← Search

Michael R. Metel

2 accepted papers

2024

Draft on the Fly: Adaptive Self-Speculative Decoding using Cosine Similarity

EMNLP 2024finding

We present a simple on the fly method for faster inference of large language models. Unlike other (self-)speculative decoding techniques, our method does not require fine-tuning or black-box optimization to generate a fixed draft model, relying instead on simple rules to generate varying draft model…

Cited by 2SourcePDFScholar