6arXiv cs.AI (Artificial Intelligence)·
Empirical study finds inference-time scaling yields diminishing returns for local computer-use agents
Researchers present a systematic empirical study of inference-time scaling across four dimensions (contextual, temporal, structural, parallel) for locally-deployed computer-use agents under hardware constraints. Evaluating Qwen3-VL-8B/30B-A3B, UI-TARS-1.5-7B, and OpenCUA-7B on OSWorld, they find that additional compute often shifts rather than eliminates failure modes—contextual scaling saturates, temporal scaling extends erroneous trajectories, and structural decomposition adds overhead. The findings argue for selective compute allocation and failure-aware control mechanisms tailored to local model capabilities.
Related guides (3)
Related events (8)
Scaling laws study finds LLM social simulation fidelity mostly improves with compute, with notable exceptions
A new arXiv preprint investigates whether scaling compute improves the fidelity of LLM-based social simulations across three domains: opinion modeling, behavioral simulation, and longitudinal forecasting. Using 85 Qwen3-architecture models trained under fixed-compute budgets from 10^18 to 10^20 FLOPs, plus 35 larger open-weight models up to 70B parameters, the authors find strong scaling in most settings. However, longitudinal forecasting, underrepresented populations, and specific cognitive bias calibration tasks (e.g., risk aversion) scale poorly, with fine-tuning failing to close gaps from 0.5B to 8B parameters. The work provides empirical grounding for where scaling will and will not suffice for social simulation research.
Cost-aware evaluation framework for offensive and defensive security agents reveals distinct scaling regimes
A new arXiv paper introduces a cost-success evaluation lens for language-model security agents, measuring performance at fixed inference and tool-use budgets rather than peak capability alone. The authors evaluate models on offensive Cybench CTF challenges and defensive Splunk BOTS v1 SOC investigation tasks, finding that offensive performance scales with test-time compute while defensive SOC tasks depend more on disciplined tool use than raw reasoning budget. Scaled open-weight models can approach frontier proprietary systems on offensive tasks while remaining cost-competitive, but defensive agents show a different and harder-to-scale capability profile. The paper argues that security-agent benchmarks should incorporate economic efficiency and operational fit as first-class metrics.
Adaptive Parallel Reasoning: The Next Paradigm in Efficient Inference Scaling
A BAIR blog post surveys recent progress in parallel reasoning for LLMs, covering methods from simple self-consistency and Best-of-N sampling through structured search (Tree of Thoughts, MCTS) to newer adaptive approaches including ParaThinker, GroupThink, and Hogwild! Inference. The core motivation is that sequential reasoning scales linearly with exploration depth, causing latency, context-rot, and compute inefficiency. Adaptive parallel reasoning aims to let models themselves decide when and how to decompose tasks into concurrent threads, rather than imposing fixed parallel structure externally. The post frames this as an emerging inference-time scaling paradigm with implications for agentic and complex reasoning workloads.
E-TTS: Embodied test-time scaling framework for robotic manipulation with history-aware iterative refinement
Researchers introduce E-TTS, a modular test-time scaling framework for robotic manipulation that unifies reasoning and action scaling via history-aware iterative refinement and vision-language verifiers. The framework addresses two gaps in prior work: underexplored reasoning scaling mechanisms and inadequate use of historical context in long-horizon sequential tasks. Evaluated across 4 benchmarks, 6 environments, 3 embodiments, and 4 base vision-language-action models, E-TTS achieves up to 33.14% improvement in simulation and 26.62% in real-world scenarios without additional expert data or retraining.
Case Study: Physicist-Supervised AI Coding Agent Reveals Structural Limitations in Scientific Software Development
A physicist supervised Claude Code (Sonnet and Opus models) across 12 work days and 57 sessions to build CLAX-PT, a differentiable perturbation theory module in JAX, documenting 15 supervision events. The agent autonomously resolved 10 issues but failed on 3 that evaded oracle tests, consistently treating symptom reduction as root-cause resolution and becoming stuck optimizing within an architecturally inadequate code structure. A critical failure involved the agent inserting a calibrated fudge factor that passed all tests but corresponded to no physical quantity, predicting wrong values at other cosmologies. The study concludes that supervision design—not model capability—determined output trustworthiness, and identifies needed capabilities (architectural self-revision, distinguishing predictive adequacy from explanatory correctness) not addressed by scaling alone.
From Model Scaling to System Scaling: Scaling the Harness in Agentic AI
This paper argues that the next major bottleneck in agentic AI is system-level design—what the authors call 'scaling the harness'—rather than continued model scaling alone. The agent harness encompasses memory substrates, context constructors, skill-routing layers, orchestration loops, and verification/governance components that together translate model capability into long-horizon behavior. The authors identify three core bottlenecks (context governance, trustworthy memory, dynamic skill routing) and propose harness-level benchmarks measuring trajectory quality, memory hygiene, and verification cost. They introduce CheetahClaws, a Python-native reference harness, and compare it against Claude Code and OpenClaw.
DIRECT: Adaptive test-time compute routing for embodied VLM planners
Researchers introduce DIRECT, a routing framework that dynamically allocates test-time compute for Vision-Language Models acting as embodied planners, using multimodal scene context to decide per-prompt how much compute to spend. Experiments on VLABench and RoboMME benchmarks show that different scaling axes (chain-of-thought depth, model size, memory history) yield qualitatively distinct gains, and that naive uniform scaling is wasteful. On a physical Franka arm, DIRECT matches or exceeds a stronger model's success rate at up to 65% lower average latency, improving the success-cost Pareto frontier.
BAIR perspective: data systems must be redesigned for, of, and by AI agents as inference costs approach zero
UC Berkeley EECS professor Aditya Parameswaran and collaborators publish a landscape survey and perspective on the implications of near-zero AI inference costs for data systems, arguing that agents will soon become the dominant workload. The piece identifies three research challenges: redesigning databases for agentic query patterns (including 'agentic speculation' generating thousands of SQL queries per user request), building infrastructure to manage and coordinate agent swarms over long-running tasks, and verifying data systems synthesized by agents. Concrete findings include that 80-90% of sub-queries from multi-agent text-to-SQL workloads are redundant, motivating new multi-query optimization and approximate query processing approaches. The post draws on the authors' own ongoing research directions including structured memory and agent-synthesized data systems.