5arXiv cs.CL (Computation and Language)·
Agora: Auction-based task allocation framework for LLM agent reasoning
Researchers introduce Agora, a framework that uses an incentive-compatible auction mechanism to dynamically route reasoning subtasks to the most capable expert models or tools, rather than relying on coarse-grained function matching. Agents bid based on 'rectified competence' to prevent overconfident solvers from capturing critical logic steps. Evaluations across five benchmarks show improvements over single-model, routing, and cascade baselines, with a controllable cost-quality trade-off via a single auction parameter.
Related guides (2)
Related events (8)
Benchmark and study of LLM agents in deliberative multi-agent collaboration under partial observability
Researchers introduce a scalable benchmark for evaluating LLM agents on cooperative joint decision-making tasks where agents must exchange information under partial and asymmetric observations to reach a shared decision. A systematic evaluation of representative LLMs finds that state-of-the-art models still struggle with complex deliberative collaboration, failing in either information alignment or downstream reasoning even with external mathematical tools. Diagnostic analysis also reveals that deliberation can enable reflection and error correction, sometimes outperforming centralized baselines, offering a nuanced picture of multi-agent LLM capabilities.
OptiAgent: Multi-agent framework for end-to-end operations research optimization modeling
OptiAgent is a multi-agent LLM framework that converts natural language descriptions of Operations Research problems into solver-ready mathematical formulations and executable code. The architecture uses dedicated agents for extracting decision variables and constraints, with a multi-loop validation system featuring four specialized feedback mechanisms targeting distinct failure modes. The system claims state-of-the-art performance on 3 of 4 benchmarks spanning LP, MILP, and Nonlinear Programming tasks, while also improving transparency through auditable agent reasoning.
Role-factorized hierarchical search agents: delegation capacity matters more than execution
A new arXiv paper investigates how model capacity should be distributed across roles in multi-agent search systems, factorizing hierarchical search into delegation, execution, and answer generation roles. Controlled sweeps across five multi-hop QA benchmarks find that scaling the delegation backbone improves exact match by ~11 points while scaling execution sub-agents yields only ~2.6 points, identifying task decomposition as the primary bottleneck. A 1.7B-parameter executor trained via trajectory distillation matches frontier sub-agent accuracy while using 37% fewer tokens, advancing the efficiency Pareto frontier. The results offer a concrete design recipe: concentrate capacity at delegation and downsize execution.
Agon: Competitive cross-model RL uses rival models as implicit reasoning graders
Agon is a new reinforcement learning framework where two competing models grade each other implicitly by attempting the same problems in alternating roles — one drafts a solution, the other reads it while solving, and each is rewarded for out-solving the rival. This sidesteps the need for process labels or a reward model, and because both models are jointly optimized, each faces a progressively stronger opponent. On the hard split of DeepMath with Qwen3, Agon doubles GRPO's pass@1, roughly eight times the gain of an untrained Mixture-of-Agents baseline, with results replicating on competitive programming and across model families.
Benchmark Agent: Autonomous system for end-to-end benchmark construction
Researchers introduce Benchmark Agent, a fully autonomous agentic system that orchestrates the complete benchmark construction pipeline — from query analysis and subtask design to data annotation and quality control. The system was used to produce 15 benchmarks spanning text understanding, multimodal understanding, and domain-specific reasoning, with evaluation via human judges, LLM-as-a-judge, and consistency checks. The work addresses two persistent problems in the field: the labor intensity of benchmark creation and rapid performance saturation after release. Code and a demo will be publicly released.
AARRI-Bench evaluates frontier LLMs and agents on granular research-intern-level tasks
Researchers introduce AARR (Act As a Real Researcher), a new benchmark series targeting whether AI agents can emulate the professionalism, thoroughness, and nuanced judgment of human researchers in granular research scenarios—not just macro-level task execution. The first benchmark, AARRI-Bench, tests frontier models and agentic harnesses, finding that even the best configuration (Mini-SWE-Agent with Claude Opus 4.7) achieves only 68.3% success, frequently missing subtle but critical details obvious to human researchers. The work argues that closing the gap requires deeper modeling of research behavior rather than more complex scaffolding.
LLawCo framework teaches embodied multi-agent LLMs to derive and follow cooperation laws
Researchers from MERL propose LLawCo (Learning Laws of Cooperation), a framework that enables embodied LLM-based agents to autonomously align with partners and task objectives in decentralized, partially observable environments. Agents reflect on past failures to extract misaligned behavioral patterns and derive high-level behavioral laws (e.g., 'Talk when necessary', 'Wait for partner'), which are incorporated into reasoning via supervised fine-tuning. The authors also introduce PARTNR-Dialog, a new large-scale multi-agent communicative planning benchmark, and report average success rate improvements of 4.5% on PARTNR-Dialog and 6.8% on TDW-MAT over state-of-the-art open-source communicative agent frameworks across four backbone LLMs.
ACTS: Agentic Chain-of-Thought Steering for efficient and controllable LLM reasoning
Researchers introduce Agentic Chain-of-Thought Steering (ACTS), a framework that formulates inference-time reasoning control as a Markov decision process, where a controller agent adaptively steers a frozen reasoner by issuing reasoning strategy directives and steering phrases at each step. The controller is initialized from synthetic steering trajectories with multi-budget augmentation and further optimized via reinforcement learning with budget-conditioned reward shaping. ACTS matches full-thinking performance with significant token savings and enables controllable accuracy-efficiency trade-offs across multiple benchmarks and reasoner models.