5arXiv cs.CL (Computation and Language)·
WIDE: Token-level dynamic width pruning framework for efficient LLM inference
WIDE is a new end-to-end differentiable framework for token-level dynamic width pruning of LLMs, enabling each token to independently select attention-head groups and FFN-channel groups at inference time. The system introduces a pruning-kernel co-design that decomposes dynamic sparsity acceleration into mask reordering and block-level skipping, achieving near-theoretical speedups of up to 1.98x for prefill and 4.95x for decoding at 50% sparsity. At that sparsity level, WIDE reports a 55.1% performance improvement over state-of-the-art dynamic depth pruning under calibration-only settings. Code is publicly released.
Related guides (2)
Related events (8)
EADP: Entropy-aware visual token pruning for efficient VLMs under dense instructions
A new arXiv preprint introduces Entropy-Aware Dense Pruning (EADP), a framework for compressing visual tokens in vision-language models (VLMs) that addresses two failure modes in existing methods: textual noise corrupting cross-modal scoring and feature fragmentation from naive Top-K selection. EADP uses statistical entropy to filter textual noise and reformulates token selection as a submodular maximization problem with a spatial prior. The authors report state-of-the-art accuracy-efficiency trade-offs on multimodal benchmarks under strict token budgets.
AdaDSF: Adaptive Depth Sparse Framework for inference-efficient LLMs without retraining
Researchers introduce AdaDSF, a method that converts pre-trained LLMs into depth-sparse models without full retraining by using cosine similarity between layer inputs and outputs to identify redundant computation. A lightweight router selects informative tokens per layer, and a feature-preserving alignment objective maintains output quality. Evaluated on GPT-NeoX and Qwen2.5, AdaDSF reduces inference FLOPs while outperforming baselines including MoD, D-LLM, and DLO under comparable sparsity budgets.
Predictor-gated bank-wise sparsity recipe for dense-to-sparse LLM upcycling from Qwen2.5-8B
A new arXiv preprint introduces a continual training recipe to convert dense LLMs into channel-sparse models without post-hoc pruning. Starting from a Qwen2.5-8B checkpoint, the method uses a low-rank predictor to gate FFN channel routing, achieving 4x sparsity in FFN intermediate activations via a bank-wise top-k rule at 32K context. The routing module is trained on the main language modeling path, making the resulting sparsity hardware-oriented rather than approximate. The authors also identify and patch a layer-local long-context failure mode on the RULER-CWE benchmark.
CLP: Lightweight collocation-length predictor achieves zero-loss multi-token inference speedup
Researchers propose CLP (Collocation-Length Predictor), a span-level decision layer for accelerating LLM inference via multi-token prediction without quality degradation. The key insight is 'Backbone-as-Architect': the backbone LM head always generates the first token while MTP heads handle only subsequent tokens, eliminating head-backbone competition that causes repetitive outputs in prior methods. CLP uses a single linear layer (~4.6K–7.7K parameters) versus 1M-parameter gate networks in prior work, achieving 1.14x–1.29x speedup on Qwen2.5 models with near-zero repetition ratio. The paper also establishes that shorter prediction horizons improve MTP head accuracy on larger models, offering a scaling-aware design principle.
Reroute: Training-free recoverable visual token routing for vision-language models
A new arXiv preprint proposes Reroute, a training-free plug-in that replaces the standard rank-and-remove visual token pruning paradigm in VLMs with a recoverable routing mechanism. Instead of permanently discarding low-ranked tokens, Reroute defers them to re-enter the candidate pool at later decoder stages, addressing the problem that token importance shifts across decoder depth. Evaluated on LLaVA-1.5 and Qwen backbones augmented with FastV, PDrop, and Nüwa pruning methods, Reroute improves grounding performance under aggressive token reduction without sacrificing general VQA accuracy. The approach preserves the theoretical compute and KV-cache budget of the underlying pruning method.
PALS: Percentile-aware per-layer sparsity improves LLM pruning on LLaMA-2 but not universally
PALS (Percentile-Aware Layerwise Sparsity) is a one-shot pruning method that assigns per-layer sparsity ratios based on the 99th percentile of activation magnitudes, bounded within ±5% of a target ratio. On LLaMA-2-7B at 50% sparsity, PALS achieves perplexity of 10.96 vs. 12.92 for uniform Wanda, a statistically significant improvement requiring no fine-tuning. However, gains are architecture-dependent: LLaMA-3-8B shows marginal improvement and Mistral-7B shows none. A notable negative finding is that gradient-based allocation performs worse than random, suggesting gradient magnitude is a poor proxy for the impact of discrete weight removal.
DWT-Fusion: Wavelet-based training-free framework for detecting LLM-generated text
Researchers introduce DWT-Fusion, a training-free method for detecting LLM-generated text that applies discrete wavelet transforms to token-level log-probability sequences from a proxy language model, capturing local and multiscale predictability patterns rather than global statistics. The framework evaluates four voting ensemble variants and is tested on HC3, M4, and MAGE benchmarks using GPT-Neo-2.7B, GPT-J-6B, Falcon-7B, and LLaMA-3-8B as proxy models. Best ensemble results achieve AUROC of 0.9919, 0.8477, and 0.7471 on the three benchmarks respectively. The approach is notable for requiring no supervised training while remaining interpretable.
DepthWeave-KV: Token-adaptive cross-layer KV cache compression for long-context inference
A new arXiv preprint introduces DepthWeave-KV, a KV cache compression method that factorizes key-value states across neighboring transformer layers using shared low-rank channel bases while retaining token-specific residuals for attention-sensitive positions. A token-conditional depth router allocates higher reconstruction rank to instruction-bearing and retrieval-critical tokens, with calibration-free online error tracking during generation. The method achieves 8.3x KV memory reduction at 64K context while maintaining near-full-cache quality on LongBench, Needle-in-a-Haystack, and L-Eval benchmarks. The work addresses a practical bottleneck in long-context inference without requiring base model retraining.