2024
DynaMo: Accelerating Language Model Inference with Dynamic Multi-Token Sampling
NAACL 2024long
Traditional language models operate autoregressively, i.e., they predict one token at a time. Rapid explosion in model sizes has resulted in high inference times. In this work, we propose DynaMo, a suite of multi-token prediction language models that reduce net inference times. Our models *dynamical…