2026
Learning To Draft: Adaptive Speculative Decoding with Reinforcement Learning
ICLR 2026poster
Speculative decoding accelerates large language model (LLM) inference by using a small draft model to generate candidate tokens for a larger target model to verify. The efficacy of this technique hinges on the trade-off between the time spent on drafting candidates and verifying them. However, curre…