5arXiv cs.CL (Computation and Language)·
WILDTRACE: Benchmark for natural evidence integration in long-context reasoning
WILDTRACE is a new benchmark of 481 tasks over 214 naturally occurring long-form documents—including technical incident reports and literary narratives—designed to test whether models can integrate evidence dispersed across distant passages by the document's own logic. The benchmark defines seven source-internal evidence geometries grounded in Pearl's causal hierarchy and multi-hop reasoning typologies, and uses a source-first construction pipeline with multi-stage validation to avoid the distributional artifacts common in existing long-context benchmarks. The work targets a gap between retrieving information and genuinely reasoning over naturally structured evidence, which the authors argue is a defining challenge for next-generation long-context systems.
Related guides (2)
Related events (8)
ReContext: Training-free recursive evidence replay improves LLM long-context reasoning
Researchers introduce RECONTEXT, a training-free inference-time method for improving long-context reasoning in LLMs. The approach uses model-internal relevance signals to build a query-conditioned evidence pool that is replayed before final generation, without modifying the original context, external memory, or context pruning. Experiments across eight long-context datasets at 128K context length show consistent improvements on Qwen3-4B, Qwen3-8B, and Llama3-8B. The authors provide a theoretical grounding via associative memory theory, framing attention as cue-trace association and replay as trace reactivation.
DeepWeb-Bench: A Hard Deep Research Benchmark Requiring Cross-Source Evidence and Long-Horizon Derivation
DeepWeb-Bench is a new benchmark designed to stress-test frontier language models on deep research tasks—open-web search, evidence collection, and multi-step derivation—where existing benchmarks have become saturated. The benchmark evaluates nine frontier models across four capability families (Retrieval, Derivation, Reasoning, Calibration) and finds that retrieval is not the primary bottleneck; derivation and calibration failures account for over 70% of errors. Strong models fail via incomplete derivation while weak models fail via hallucinated precision, and models show genuine domain specialization with low cross-model agreement (rho = 0.61). The benchmark, rubrics, and evaluation code are publicly released.
LongTraceRL: Reinforcement Learning for Long-Context Reasoning via Search Agent Trajectories and Rubric Rewards
LongTraceRL is a new RL training framework for improving long-context reasoning in LLMs, addressing limitations of existing RLVR methods. It constructs challenging training data using multi-hop questions from knowledge graph random walks and tiered distractors derived from search agent trajectories (high-confusability: read but uncited; low-confusability: seen but unopened). A rubric reward provides entity-level process supervision along reasoning chains, applied only to correct responses to prevent reward hacking. Experiments across three LLMs (4B–30B parameters) on five long-context benchmarks show consistent improvements over strong baselines.
VeriTrace: Cognitive-Graph Framework with Explicit Regulatory Loops for Deep Research Agents
VeriTrace introduces a cognitive-graph framework for deep research agents that replaces implicit LLM reasoning over intermediate representations with three explicit regulatory loops: interpretive update, deviation feedback, and schema revision. The system addresses contamination and error propagation in evolving mental models during complex multi-step research tasks. Using Qwen3.5-27B backbones, VeriTrace improves over the strongest matched baseline by 4.22 pp on DeepResearch Bench Insight and 5.9 pp Overall win rate on DeepConsult. With Config-DeepSeek, it achieves the strongest reproducible open-source result on DeepResearch Bench.
LongCrafter: Evidence-graph-guided synthesis framework for long-context SFT data
LongCrafter is a structured framework for synthesizing long-context supervised fine-tuning data, addressing limitations of prior approaches including narrow task coverage, low difficulty, and lack of faithfulness supervision. The system uses a hierarchical 32-task taxonomy and constructs explicit evidence graphs modeling cross-paragraph dependencies to generate grounded instruction-response pairs. Models fine-tuned on LongCrafter data outperform SFT baselines and official post-trained models on LongBench, LongBench v2, and LooGLE for both Qwen2.5-7B and LLaMA-3.1-8B, with notable gains on high-difficulty tasks and improved robustness to the 'lost in the middle' problem.
DocTrace: Structure-Aware On-Demand Hypergraph Memory for Long-Document QA
Researchers introduce DocTrace, a multi-agent RAG framework for long-document question answering that uses query-triggered knowledge organization rather than costly query-agnostic preprocessing. The system combines a lightweight document structural tree index, on-demand hypergraph working memory, and a graph-structured experience memory that stores successful reasoning plans for reuse. Evaluated on four long-document QA datasets, DocTrace outperforms the strongest baseline (ComoRAG) by up to 8.85% F1 and 4.40% EM while reducing computational cost by 53.32%.
AGORA benchmark tests agentic document reasoning over large authentic workplace archives
Researchers introduce AGORA, a benchmark pairing 362 questions with 9,664 authentic workplace documents (372M tokens across eight domain collections) to evaluate archive-grounded agentic reasoning. The benchmark is designed so documents far exceed any model's context window, forcing deliberate exploration rather than exhaustive scanning. Evaluating eight models, the best achieves only 59.4% accuracy, indicating the task is far from solved. The benchmark addresses a gap in existing evals that do not jointly stress archive-groundedness, agentic exploration, and cross-domain coverage.
ReasoningLens: Open-source framework for hierarchical visualization and auditing of LLM reasoning chains
ReasoningLens is an open-source framework for visualizing and diagnostically auditing the long chain-of-thought traces produced by large reasoning models. It structures traces into interactive hierarchies separating high-level strategy from low-level execution, uses an agentic auditor for automated error detection, and synthesizes model-specific reasoning profiles to surface blind spots. The work targets a growing transparency problem as reasoning models produce increasingly long and opaque inference traces.