Entity · paper

A Practical Investigation of Training-free Relaxed Speculative Decoding

paperactiveprovisionala-practical-investigation-of-training-free-relaxed-speculative-decoding-6b567927·1 events·first seen 3d ago

Aliases: A Practical Investigation of Training-free Relaxed Speculative Decoding

Co-occurring entities

speculative decoding

More like this (12)

speculative decoding When Is a Draft Accepted? A Theory of Acceptance in Speculative Decoding VIA-SD: Verification via Intra-Model Routing for Speculative Decoding Grammar-Constrained Decoding blockwise decoding lambda-scaled structural decoding ExpRL: Exploratory RL for LLM Mid-Training Minimum Bayes Risk Decoding Noisy-Channel Minimum Bayes Risk Decoding Continual LLM Upcycling: A Predictor-Gated Bank-Wise Sparsity Training Recipe for Dense-to-Sparse LLMs Pessimism's Paradox: Conservative Offline Training Amplifies Reward Hacking During Online Adaptation in Reasoning Models RefDecoder

Recent events (1)

4arXiv · cs.AI·3d ago·source ↗

Practical investigation of training-free relaxed speculative decoding techniques

A new arXiv preprint systematically investigates training-free relaxed speculative decoding methods, which trade the strict distribution-preservation guarantee of standard speculative decoding for potential speed or capability gains. The authors unify existing approaches in a shared framework, benchmark them on contemporary settings, and surface practical findings for practitioners. Key takeaways include that relaxed approaches require careful capability evaluation and tend to depend on high-quality drafter models, making them poorly suited for lightweight multi-token-prediction drafters.

Inference Economics speculative decoding A Practical Investigation of Training-free Relaxed Speculative Decoding