adaflash-e293d2e5·1 events·first seen Aliases: AdaFlash
AdaFlash is a new framework for accelerating LLM inference via speculative decoding that addresses variance problems in diffusion-based draft models. It combines an on-policy distillation algorithm using reverse-KL divergence with an adaptive length head that dynamically adjusts candidate sequence length. Experiments show up to ~66% higher throughput than prior state-of-the-art methods, with especially large gains in high-concurrency deployment scenarios.