Entity · benchmark
LongBench v2
benchmarkactive
longbench-v2-5405e812·5 events·first seen Aliases: LongBench v2, LongBench, LongBench-v2
Co-occurring entities
DeepSeek V4GLM-5.1DeepSeek Sparse AttentionPIVOTRULERQwen3-4B-Thinking-2507Self-Guided Test-Time TrainingLlama3-8B-InstructLongBench-ProDepthWeave-KVL-EvalNeedle-in-a-HaystackQwen2.5-7BLongCrafterLooGLELlama-3.1-8BOn-Policy Distillation (OPD)Multi-Token Prediction (MTP)speculative decodingPair-In, Pair-Out (PIPO)
More like this (12)
Recent events (5)
PIVOT: Training-free sparse attention indexer cuts DeepSeek-V3.2 latency by up to 1.6x
PIVOT (Proxy Indexing Via One full-prefix Traversal) is a training-free drop-in replacement for the DeepSeek Sparse Attention (DSA) indexer that reduces the O(L²) per-query scan cost by grouping nearby queries and sharing a single prefix scan across the group. Two variants (PIVOT-Reuse and PIVOT-Refine) trade speed for fidelity, with PIVOT-Refine matching dense indexer accuracy. Evaluated on DeepSeek-V3.2 and GLM-5.1 across LongBench and RULER, PIVOT accelerates the indexer by up to 4x and reduces end-to-end latency by up to 1.6x at long context.
Self-Guided Test-Time Training improves long-context LLM utilization by up to 15%
Researchers propose Self-Guided Test-Time Training (S-TTT), a method that addresses the degradation in accuracy LLMs exhibit on long inputs by having the model first identify relevant evidence spans before applying test-time training only to those spans. A preliminary study on LongBench-v2 shows that TTT on randomly sampled spans hurts performance while TTT on oracle spans substantially helps, motivating the self-guided selection approach. Evaluated on LongBench-v2 and LongBench-Pro with Qwen3-4B-Thinking-2507 and Llama-3.1-8B-Instruct, S-TTT achieves up to 15% relative accuracy improvement. The method offers a practical path to better long-context utilization without the prohibitive cost of full-context adaptation.
DepthWeave-KV: Token-adaptive cross-layer KV cache compression for long-context inference
A new arXiv preprint introduces DepthWeave-KV, a KV cache compression method that factorizes key-value states across neighboring transformer layers using shared low-rank channel bases while retaining token-specific residuals for attention-sensitive positions. A token-conditional depth router allocates higher reconstruction rank to instruction-bearing and retrieval-critical tokens, with calibration-free online error tracking during generation. The method achieves 8.3x KV memory reduction at 64K context while maintaining near-full-cache quality on LongBench, Needle-in-a-Haystack, and L-Eval benchmarks. The work addresses a practical bottleneck in long-context inference without requiring base model retraining.
LongCrafter: Evidence-graph-guided synthesis framework for long-context SFT data
LongCrafter is a structured framework for synthesizing long-context supervised fine-tuning data, addressing limitations of prior approaches including narrow task coverage, low difficulty, and lack of faithfulness supervision. The system uses a hierarchical 32-task taxonomy and constructs explicit evidence graphs modeling cross-paragraph dependencies to generate grounded instruction-response pairs. Models fine-tuned on LongCrafter data outperform SFT baselines and official post-trained models on LongBench, LongBench v2, and LooGLE for both Qwen2.5-7B and LLaMA-3.1-8B, with notable gains on high-difficulty tasks and improved robustness to the 'lost in the middle' problem.
Pair-In, Pair-Out (PIPO): Unified Latent Compression and Multi-Token Prediction for Efficient LLM Inference
PIPO is a new inference efficiency framework that unifies input-side latent compression with output-side multi-token prediction (MTP) by treating them as mirror operations: a compressor folds two input tokens into one latent, while an MTP head unfolds one hidden state into an additional output token. To avoid the expensive verifier pass typically required by speculative decoding, PIPO trains a lightweight confidence head using On-Policy Distillation (OPD), which naturally aligns with rejection-sampling criteria. Experiments on Qwen3.5-4B and 9B backbones across AIME 2025, GPQA-Diamond, LiveCodeBench v6, and LongBench v2 show up to 2.64× first-token-latency speedup and +7.15 pass@4 improvement over regular decoding.