6arXiv cs.AI (Artificial Intelligence)·
Fluid search: adaptive budget allocation for efficient autonomous research systems
A new arXiv paper argues that autonomous research (AR) systems should be evaluated not only on final outcome quality but also on search efficiency — how quickly they reach good solutions within a fixed evaluation budget. The authors propose using the AUC of the Pareto frontier as an efficiency metric and compare hill climbing, beam search, tree search, and evolutionary search across twelve systems-optimization tasks, finding no single algorithm dominates. They introduce 'fluid search,' a portfolio bandit method that dynamically allocates budget across a forest of search processes, achieving oracle-level efficiency without knowing the best search structure in advance.
Related guides (2)
Related events (8)
Systematic evaluation finds no universally superior autonomous discovery harness; adaptive allocation outperforms fixed choices
A new arXiv preprint systematically decomposes OpenEvolve-style evolutionary search and TTT-Discover into constituent components, evaluating 30 budget-matched harnesses across 12 model-problem pairs using over 3.1 million LLM rollouts. Results show no fixed harness generalizes reliably across settings, and simpler alternatives often outperform OpenEvolve variants. The authors demonstrate that early discovery progress predicts final performance and use this to motivate an adaptive budget-allocation strategy that prunes weak runs and reallocates compute to stronger survivors, outperforming fixed harness selection.
Role-factorized hierarchical search agents: delegation capacity matters more than execution
A new arXiv paper investigates how model capacity should be distributed across roles in multi-agent search systems, factorizing hierarchical search into delegation, execution, and answer generation roles. Controlled sweeps across five multi-hop QA benchmarks find that scaling the delegation backbone improves exact match by ~11 points while scaling execution sub-agents yields only ~2.6 points, identifying task decomposition as the primary bottleneck. A 1.7B-parameter executor trained via trajectory distillation matches frontier sub-agent accuracy while using 37% fewer tokens, advancing the efficiency Pareto frontier. The results offer a concrete design recipe: concentrate capacity at delegation and downsize execution.
Transformer-guided swarm intelligence framework for low-resource neural architecture search
A new arXiv preprint proposes a hybrid NAS framework combining an autoregressive Transformer controller trained via reinforcement learning with an Artificial Bee Colony metaheuristic for local search. The system targets consumer-grade hardware (NVIDIA RTX 3060), completing architecture search in ~3 hours and finding compact models: 84.85% on CIFAR-10 with ~174K parameters and F1=0.71 on fraud detection with ~4,600 parameters. A dynamic entropy mechanism is introduced to prevent premature convergence during RL training. The work addresses the democratization angle of NAS, positioning results as suitable for edge deployment.
IMPFM: Multi-particle flow-map framework for online feedback-driven generative search
A new arXiv preprint introduces Sequentially-Controlled Interactive Multi-Particle Flow-Maps (IMPFM), a framework for sample-efficient online search under sequential human feedback. The method transports an ensemble of particles toward a target distribution using flow maps and a posterior sample-sharing mechanism, aiming to balance global exploration with reward over-optimization avoidance. The authors prove convergence to a KL-tilted target via a Feynman-Kac corrector and demonstrate improvements over baselines on alignment and search tasks. The work addresses a gap in training-free reward alignment methods that struggle with unknown or sequentially revealed preferences.
AutoLab benchmark evaluates frontier models on ultra long-horizon iterative research and engineering tasks
AutoLab is a new benchmark of 36 expert-curated tasks across system optimization, puzzle-solving, model development, and CUDA kernel optimization, designed to test agents on sustained closed-loop improvement under wall-clock budgets rather than single-turn or short-horizon settings. Evaluation of 17 frontier models finds that persistence in iterative benchmarking and feedback incorporation — not initial attempt quality — is the dominant success predictor. Claude Opus 4.6 stands out as the strongest performer, while most models including proprietary ones either terminate early or exhaust budgets with minimal progress. The benchmark, harness, and task artifacts are open-sourced.
Shadow evaluations show frontier AI agents can do AI research engineering but fail at open-ended scientific reasoning
A new arXiv preprint introduces 'shadow evaluations' — a methodology where AI agents tackle the central research question of unpublished NeurIPS 2026 papers, with original authors grading the output. Frontier agents given six days and thousands of dollars of compute completed all engineering tasks without human help but failed to make substantive scientific progress, resulting in both papers being rejected by their authors. The authors identify five recurring failure modes including poor judgment about publishability, uncreative responses to design shortcomings, and instruction drift. The work provides early empirical evidence that the engineering-vs.-research gap is a real bottleneck for AI R&D automation.
Agora: Auction-based task allocation framework for LLM agent reasoning
Researchers introduce Agora, a framework that uses an incentive-compatible auction mechanism to dynamically route reasoning subtasks to the most capable expert models or tools, rather than relying on coarse-grained function matching. Agents bid based on 'rectified competence' to prevent overconfident solvers from capturing critical logic steps. Evaluations across five benchmarks show improvements over single-model, routing, and cascade baselines, with a controllable cost-quality trade-off via a single auction parameter.
Cost-aware evaluation framework for offensive and defensive security agents reveals distinct scaling regimes
A new arXiv paper introduces a cost-success evaluation lens for language-model security agents, measuring performance at fixed inference and tool-use budgets rather than peak capability alone. The authors evaluate models on offensive Cybench CTF challenges and defensive Splunk BOTS v1 SOC investigation tasks, finding that offensive performance scales with test-time compute while defensive SOC tasks depend more on disciplined tool use than raw reasoning budget. Scaled open-weight models can approach frontier proprietary systems on offensive tasks while remaining cost-competitive, but defensive agents show a different and harder-to-scale capability profile. The paper argues that security-agent benchmarks should incorporate economic efficiency and operational fit as first-class metrics.