DDTree

DominoTree: Training-free conditional tree drafting for speculative decoding achieves 6.6x LLM inference speedup

DominoTree is a new training-free speculative decoding method that constructs best-first draft trees scored by Domino's conditional, non-factorized GRU-based correction along each root-to-node path. On Qwen3-4B across eight benchmarks, it achieves up to 6.6x speedup over autoregressive decoding and a mean accept length of up to 10.7 tokens per round, outperforming prior methods including DDTree, CaDDTree, DFlash, and the base Domino decoder. A GPU-native CUDA-graph tree builder provides 9-10% throughput gains over Domino overall, with up to +22% on Alpaca, while maintaining bit-identical acceptance behavior.