2026
Scaling Behavior of Discrete Diffusion Language Models
Dimitri von Rütte, Antonio Orvieto, Janis Fluri, Omead Pooladzandi, Bernhard Schölkopf, Thomas Hofmann
ICLR 2026poster
Modern LLM pre-training consumes vast amounts of compute and training data, making the scaling behavior, or scaling laws, of different models a key distinguishing factor. Discrete diffusion language models (DLMs) have been proposed as an alternative to autoregressive language models (ALMs). However,…