Zihao An

PARD: Accelerating LLM Inference with Low‑Cost PARallel Draft Model Adaptation

The autoregressive nature of large language models (LLMs) fundamentally limits inference speed, as each forward pass generates only a single token and is often bottlenecked by memory bandwidth. Speculative decoding has emerged as a promising solution, adopting a draft-then-verify strategy to acceler…