4arXiv cs.CL (Computation and Language)·
DeLS-Spec: Decoupled long-short context speculative decoding improves drafting efficiency
DeLS-Spec is a new speculative decoding method that combines a fixed block-parallel draft model (DFlash) as a long-context expert with a lightweight locally-trained short-context head, avoiding joint training with the target model. The approach introduces intra-block causal conditioning at low training cost and is modular across DFlash checkpoints. Experiments on Qwen3 models show consistent speedup and acceptance-length improvements over DFlash on math, code, and dialogue benchmarks.
Related guides (1)
Related events (8)
SimSD: Speculative Decoding Adapted for Diffusion Language Models
SimSD introduces a training-free speculative decoding algorithm for diffusion large language models (dLLMs), which previously could not use standard token-level speculative decoding due to their bidirectional attention and masked language modeling formulation. The method uses a plug-and-play masking strategy that introduces reference tokens from a draft model and a custom attention mask, enabling valid logit computation for drafted tokens in a single forward pass. Evaluated on SDAR-family dLLMs across four benchmarks, SimSD achieves up to 7.46x decoding throughput improvement while maintaining or improving generation quality. The approach is compatible with other acceleration techniques such as KV cache and blockwise decoding.
DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation for LLM Inference
DSpark is a new speculative decoding framework that combines a semi-autoregressive draft architecture with confidence-scheduled verification to improve LLM inference throughput. The semi-autoregressive design introduces intra-block token dependencies to reduce acceptance decay common in parallel drafters, while dynamic verification length tuning reduces wasted batch capacity in high-concurrency settings. Deployed within the DeepSeek-V4 serving system under live traffic, DSpark achieves 60–85% faster per-user generation speeds compared to the MTP-1 production baseline at matched throughput, and expands the Pareto frontier of latency-throughput tradeoffs.
VIA-SD: Multi-tier speculative decoding via intra-model routing cuts rejection rates and boosts inference speed
VIA-SD introduces a three-tier verification framework for speculative decoding that routes draft tokens to a lightweight 'slim verifier' submodel for medium-confidence cases, reserving full-model verification only for uncertain tokens. Across four tasks and multiple model families, the method reduces rejection rates by 0.10–0.22 and achieves 10–20% speedups over strong speculative decoding baselines, with 2.5–3x acceleration over standard decoding. The approach is compatible with existing speculative decoding frameworks without retraining. The work proposes multi-tier speculative decoding as a general paradigm for scalable LLM inference.
DeepSeek releases DSpark: speculative decoding system for LLM inference acceleration
DeepSeek published DSpark, a paper describing a speculative decoding system designed to accelerate LLM inference. The paper is hosted on DeepSeek's GitHub and attracted significant Hacker News engagement (598 points, 228 comments), suggesting meaningful community interest. Speculative decoding is an active inference optimization technique, and a release from DeepSeek carries weight given their track record on inference efficiency.
Faster Text Generation with Self-Speculative Decoding via LayerSkip
This Hugging Face blog post covers LayerSkip, a self-speculative decoding technique that accelerates text generation by using early exit from transformer layers to draft tokens, then verifying them with the full model. Unlike standard speculative decoding, LayerSkip requires no separate draft model, reducing memory overhead while still achieving inference speedups. The post likely covers integration with the Hugging Face ecosystem and practical performance benchmarks.
Faster Assisted Generation with Dynamic Speculation
Hugging Face introduces dynamic speculation lookahead for assisted (speculative) decoding, a technique that adaptively adjusts the number of candidate tokens generated by a draft model before verification by the main model. This approach aims to improve throughput and reduce latency compared to fixed-lookahead speculative decoding by tuning the speculation depth at runtime. The blog post describes the method and its integration into the Hugging Face Transformers library.
ADAS: Attention-Discounted Adaptive Sampler improves parallel decoding for masked diffusion language models
Researchers propose ADAS, a training-free reranking rule for masked diffusion language model decoding that addresses token interaction failures in parallel token commitment. The method greedily penalizes candidates that attend strongly to already-selected uncertain positions, using attention weights as soft marginal penalties rather than hard constraints. Evaluated on LLaDA-8B-Base and Dream-7B-Base across GSM8K, MATH500, HumanEval, and MBPP, ADAS improves low-NFE performance by 9–10 percentage points on average when plugged into existing samplers with only 3.1% runtime overhead.
Accelerating Qwen3-8B Agent on Intel Core Ultra with Depth-Pruned Draft Models
Hugging Face and Intel demonstrate speculative decoding acceleration for the Qwen3-8B model on Intel Core Ultra client hardware using depth-pruned draft models. The approach applies structured pruning to create a smaller draft model that enables speculative decoding, targeting on-device agent workloads. This work addresses inference efficiency for mid-size open-weight models on consumer-grade x86 silicon.