2026
Probability Distributions Computed by Autoregressive Transformers
ICLR 2026poster
Most expressivity results for transformers treat them as language recognizers (which accept or reject strings), and not as they are used in practice, as language models (which generate strings autoregressively and probabilistically). We characterize the probability distributions that transformer lan…