4arXiv cs.CL (Computation and Language)·
DAIS: Dependency-aware intermediate QA supervision improves complex reasoning in LLMs
Researchers introduce DAIS (Dependency-Aware Intermediate QA Supervision), a training-time framework that converts teacher rationales into stage-level QA records where each intermediate step is conditioned on prior reasoning states. Evaluated on GDPR, AIACT, MedQA, and FOLIO benchmarks using Qwen backbones, DAIS outperforms answer-only, flat chain-of-thought, and independent-QA baselines, with up to 5.6% and average 4.2% gains on policy-compliance tasks. Ablations confirm that dependency conditioning contributes beyond simply adding more intermediate text, suggesting it as a lightweight auxiliary supervision signal.
Related guides (3)
Related events (8)
Self-generated QA supervision is fragile: coverage bias and instruction injection in synthetic training data
A new arXiv paper demonstrates that synthetic QA generation for model training is not neutral preprocessing but an implicit policy with two critical failure modes: coverage bias (generators concentrate on salient spans, causing uneven document coverage) and instruction compliance (models answering their own generated questions tend to obey instruction-like passages embedded in source text, with 88% compliance rate). The authors show that larger models exhibit worse instruction injection compliance, and propose two mitigations—anchoring questions to fixed target spans and filtering instruction-like passages before answering—reducing injection compliance from 88% to 13% while preserving clean-text quality. This has direct implications for synthetic data pipelines, knowledge distillation, and document compression workflows widely used across the field.
ExpRL: RL-based mid-training using human QA data as reward scaffolds for LLM reasoning
ExpRL proposes an automated approach to LLM mid-training that replaces manually curated reasoning traces with large corpora of human-written QA data used as reward scaffolds rather than imitation targets. Reference solutions are hidden from the policy and used only to construct problem-specific grading rubrics, enabling dense process-level rewards that reinforce partial progress and intermediate reasoning steps. On challenging math reasoning benchmarks, ExpRL outperforms SFT, sparse-reward GRPO, and self-distillation as an RL initialization strategy, with additional mixed-domain experiments suggesting broader applicability.
Question-Answering as Hidden State Probing for Test-Time Reasoning Intervention
This paper proposes using question-asking as an inference-time intervention to surface information about an LLM's hidden state during chain-of-thought reasoning. The authors train a probe on a student model's hidden states before and after question generation, finding it predictive of final answer correctness even before the teacher responds—suggesting self-diagnosis during question generation carries meaningful signal. They frame question-asking as a sequential decision problem with a gating policy, but find a gap between detection and recovery: interventions are as likely to harm correct trajectories as to fix incorrect ones. The results have implications for the limits of LLM self-refinement under uncertainty.
QVal: Training-free benchmark for evaluating dense supervision signals in long-horizon LLM agents
QVal is a new training-free testbed for evaluating dense supervision signals used to guide LLM agents over long-horizon trajectories, where outcome-only rewards are too sparse. The framework measures 'Q-alignment' — whether a method's step scores match Q-values from a strong reference policy — enabling comparison of 21 methods across 4 environments and 7 methodological families without running full training pipelines. A key finding is that simple prompting baselines consistently outperform more sophisticated dense supervision methods from recent literature. The benchmark covers over 1,200 evaluation experiments across six open-weight model backbones.
ModSleuth: Agentic system audits invisible dependency graphs in modern LLM training pipelines
Researchers introduce ModSleuth, an agentic system that recursively reconstructs LLM dependency graphs from public artifacts, recovering 1,060 source-verified dependencies across four major LLM releases. The system formalizes direct and indirect dependencies and operation-centered relationships to handle fragmented, inconsistent documentation. Applied at scale, the resulting graphs expose multi-hop license obligations, train-evaluation coupling, and discrepancies between released and training-time artifacts — issues that are practically invisible to manual auditing.
SelectBench and DAPO post-training for selective evidence adoption in RAG contexts
Researchers introduce SelectBench, a benchmark and training set for evaluating whether retrieval-augmented LLMs can selectively adopt valid evidence while rejecting misleading or injected content. They post-train Qwen3.5-4B using DAPO with rule-based and semantic judge rewards, achieving modest but directional improvements on SelectBench-v2 (22.46% to 26.46% strict success). Gains do not survive Holm multiple-comparison correction, and prompt-injection resistance shows no improvement, leaving statistical robustness and injection resistance as open challenges. General capabilities on MMLU and HotpotQA are preserved.
Question-type-aware LLM pipeline with multi-agent collaboration achieves top factoid performance on BioASQ 14b
Researchers present a question-type-specific LLM framework for the BioASQ 14b biomedical QA challenge, applying distinct inference strategies to yes/no, factoid, and list questions rather than a single unified approach. For list questions, a multi-agent architecture handles evidence extraction, candidate generation, verification, and aggregation collaboratively. The framework achieved first place in the factoid subtask of Batch 4 in the official BioASQ 14b evaluation, demonstrating competitive performance across multiple batches.
MADA-RL: Multi-agent debate with counterfactual critic advantage for parameter-efficient reasoning in compact models
MADA-RL is a post-training framework that specializes compact LLMs (≤4B parameters) into generator and critic roles, training them with a debate-aware RL signal using LoRA adapters on only a small fraction of parameters. The central contribution is a counterfactual critic advantage that conditions the critic's reward on the generator ensemble's per-instance accuracy, explicitly incentivizing critics to correct generator errors rather than reproduce correct answers. Applied to DeepSeek-R1-Distill-Qwen-1.5B, the method achieves +2.0 percentage points on mathematical reasoning benchmarks using 16× fewer trainable parameters than fully fine-tuned baselines, placing it on the accuracy-vs-trainable-parameter Pareto front. The approach does not surpass the strongest baselines (DeepScaleR, STILL-3) trained on larger datasets, and the paper analyzes this gap directly.