4arXiv cs.CL (Computation and Language)·
Task-specific two-agent architecture wins QANTA 2026 multimodal QA challenge
Researchers present the top-scoring submission to the QANTA 2026 shared challenge at ICML 2026's EMM-QA Workshop, achieving an overall leaderboard score of 0.402 on multimodal quizbowl tasks. The system uses a two-agent architecture: a GPT-4.1-mini-based Tossup agent with confidence calibration and a GPT-4.1-based Bonus agent with structured relational and multimodal reasoning. Notably, the approach avoids retrieval pipelines and model ensembles, relying instead on lightweight task-specific reasoning policies under efficiency constraints. Results suggest that targeted reasoning strategies can be competitive on resource-constrained multimodal QA benchmarks.
Related guides (4)
Related events (8)
Question-type-aware LLM pipeline with multi-agent collaboration achieves top factoid performance on BioASQ 14b
Researchers present a question-type-specific LLM framework for the BioASQ 14b biomedical QA challenge, applying distinct inference strategies to yes/no, factoid, and list questions rather than a single unified approach. For list questions, a multi-agent architecture handles evidence extraction, candidate generation, verification, and aggregation collaboratively. The framework achieved first place in the factoid subtask of Batch 4 in the official BioASQ 14b evaluation, demonstrating competitive performance across multiple batches.
Benchmark Agent: Autonomous system for end-to-end benchmark construction
Researchers introduce Benchmark Agent, a fully autonomous agentic system that orchestrates the complete benchmark construction pipeline — from query analysis and subtask design to data annotation and quality control. The system was used to produce 15 benchmarks spanning text understanding, multimodal understanding, and domain-specific reasoning, with evaluation via human judges, LLM-as-a-judge, and consistency checks. The work addresses two persistent problems in the field: the labor intensity of benchmark creation and rapid performance saturation after release. Code and a demo will be publicly released.
OmniAgent: POMDP-based active perception agent for long video understanding with test-time scaling
Researchers introduce OmniAgent, a multimodal agent that reformulates long video understanding as a POMDP-based iterative Observation-Thought-Action cycle, selectively distilling audio-visual cues into persistent textual memory rather than processing all frames uniformly. The system uses Agentic Supervised Fine-Tuning and a novel reinforcement learning method (TAURA) with turn-level entropy for credit assignment. OmniAgent demonstrates positive test-time scaling and achieves state-of-the-art open-source results across ten benchmarks, with its 7B model outperforming Qwen2.5-VL-72B on LVBench (50.5% vs. 47.3%).
T1-Bench: Multi-scenario agent benchmark across 25 real-world domains
T1-Bench is a new benchmark for evaluating agentic LLM systems in realistic customer-facing, multi-domain environments, covering 25 domains of varying difficulty with interleaved multi-turn scenarios. The authors evaluate 12 proprietary and open-weight models and combine automatic evaluation with human judgments. The benchmark targets gaps in existing agent evals around task complexity, domain diversity, and compositional reasoning across multi-step interactions.
Thinking Machines Lab Reveals TML-Interaction-Small: Real-Time Multimodal Interaction Model
Thinking Machines Lab (founded by Mira Murati) has announced TML-Interaction-Small, a 276B-parameter mixture-of-experts multimodal model that processes audio, video, and text concurrently using 200ms 'micro-turns' rather than waiting for conversational turns to complete. The architecture uses encoder-free early fusion, pairing a fast foreground interaction model with an asynchronous background reasoning model that shares context. On interactivity benchmarks (FD-bench V1/V1.5), it outperforms GPT-Realtime-2 and Gemini-3.1-flash-live-preview, though it trails GPT-Realtime-2 on intelligence benchmarks. A closed research preview is expected in coming months with wider release later in 2026.
M³Exam: Benchmark for Multimodal Memory in Realistic User-Agent Interactions
Researchers introduce M³Exam, a query-centric multimodal conversational memory benchmark designed to evaluate language agents on realistic user-agent interactions, including cross-modal grounding and implicit information inference. Existing benchmarks are critiqued for assuming sparse visuals and human-human interaction formats. The paper also proposes M³Proctor, a companion memory method that detects query modality bias and retrieves raw visual sources on demand, achieving 13% accuracy improvement while reducing index-construction time and retrieved tokens by over 70%.
Multi-agent system using open-source LLMs outperforms GPT-4 on disinformation detection
A new arXiv preprint proposes a multi-agent system for automated disinformation detection that emulates human annotator decision-making through consensus mechanisms, cognitive diversity, and hierarchical structure. The system uses open-source models (LLaMA, Kimi, Qwen, DeepSeek, LLaMA-Nemotron) and is evaluated on English, Polish, Slovak, and Bulgarian datasets across three fact-checking tasks. Results claim superior performance over individual LLMs including GPT-4 and GPT-3.5, with transparency benefits from using open weights models.
NCP-ExploreToM benchmark evaluates LLMs' ability to induce belief states through action rather than conversation
Researchers introduce Non-Conversational Planning Theory of Mind (NCP-ToM) and a novel evaluation framework, NCP-ExploreToM, which tests whether LLMs can manipulate belief states in other agents by moving objects or directing characters rather than through dialogue. Six frontier models (including GPT-5, Gemini 2.5 Pro, and Claude 4 series) and human participants were evaluated across 600 task instances; GPT-5 achieved ~80% success and was the only model to outperform humans, though it remained less robust across contexts. All models, like humans, performed better at inducing true belief states than false ones, which the authors flag as a positive alignment signal. The work highlights both emerging agentic social-reasoning capabilities and new manipulation/misinformation risks that passive ToM benchmarks fail to capture.