AdaSpec: Adaptive Multilingual Speculative Decoding with Self-Synthesized Language-Aware Training and Vocabulary Simplification
Dinh-Truong Do, Nguyen-Khang Le, Le-Minh Nguyen
Abstract
Speculative decoding accelerates large language model (LLM) inference by using a lightweight drafter to propose multiple tokens, which are then verified in parallel by the base model. While effective in English, existing methods often struggle in multilingual scenarios due to static vocabularies and the lack of language-specific instruction data. To address these limitations, we present AdaSpec, a multilingual speculative decoding framework that dynamically adapts both the drafter and vocabulary at decoding time. AdaSpec generates language-specific instruction data using the LLM itself, enabling training of drafters for low-resource languages. It also constructs adaptive vocabularies tailored to each language
BibTeX
@inproceedings{aaai2026_adaspecadaptivem,
title = {AdaSpec: Adaptive Multilingual Speculative Decoding with Self-Synthesized Language-Aware Training and Vocabulary Simplification},
author = {Dinh-Truong Do and Nguyen-Khang Le and Le-Minh Nguyen},
booktitle = {AAAI 2026},
year = {2026}
}