eit-nlp-4a39b273·4 events·first seen Aliases: EIT-NLP
WIDE is a new end-to-end differentiable framework for token-level dynamic width pruning of LLMs, enabling each token to independently select attention-head groups and FFN-channel groups at inference time. The system introduces a pruning-kernel co-design that decomposes dynamic sparsity acceleration into mask reordering and block-level skipping, achieving near-theoretical speedups of up to 1.98x for prefill and 4.95x for decoding at 50% sparsity. At that sparsity level, WIDE reports a 55.1% performance improvement over state-of-the-art dynamic depth pruning under calibration-only settings. Code is publicly released.
A new arXiv paper introduces a three-stage calibration framework analyzing how supervised fine-tuning (SFT), reinforcement learning (RL), and on-policy distillation (OPD) shape model confidence before, during, and after chain-of-thought reasoning. The authors find that each post-training method produces distinct calibration profiles at different reasoning stages, and that RL confidence becomes informative only after a path-commitment phase while OPD confidence degrades later. They propose PosConf, a position-aware confidence strategy that selectively uses confidence from reliable relative-position intervals, improving RL answer aggregation by 6.1 points over majority voting and OPD early stopping by up to 4.3 points.
This paper analyzes Latent Chain-of-Thought (CoT) reasoning — where reasoning occurs in continuous hidden states rather than discrete text — through an information-theoretic lens, identifying a 'dual collapse' failure mode involving gradient attenuation and representational drift. The authors decompose process supervision into Trajectory Supervision and Space Supervision, and introduce the Unified Latent Probe (ULP) to quantify mutual information between latent trajectories and explicit reasoning steps. Experiments reveal an 'Information-Performance Binding' showing reasoning accuracy depends on information fidelity in the latent chain, suggesting supervision should shift from geometric imitation toward mutual information maximization.
Researchers introduce AdaSR, a framework enabling large reasoning models to reason incrementally during streaming input (e.g., audio/video) rather than waiting for complete context, then perform final deliberation once the stream ends. The core contribution is Hierarchical Relative Policy Optimization (HRPO), which decomposes policy optimization into streaming and deep reasoning phases with fine-grained per-phase advantage assignment, integrating format, accuracy, and latency-aware rewards. Experiments show AdaSR improves the tradeoff among reasoning accuracy, computational efficiency, and streaming latency over supervised fine-tuning baselines. Code is publicly released.