SpecBranch: Speculative Decoding via Hybrid Drafting and Rollback-Aware Branch Parallelism
Speculative decoding (SD) has emerged as a promising technique to accelerate LLM inference by employing a small, efficient draft model to propose draft tokens in advance, and subsequently validating them in parallel with the large target model. However, the existing SD methods still remain fundament…