2026
Prototype Transformer: Towards Language Model Architectures Interpretable by Design
Yordan Yordanov, Matteo Forasassi, Bayar Menzat, Tommaso Salvatori, Ruizhi Wang, Markus Kaltenberger +3
ICML 2026poster
While state-of-the-art language models (LMs) surpass the vast majority of humans in certain domains, their reasoning remains largely opaque, reducing trust and risking deception and hallucination. In this work, we introduce the Prototype Transformer (ProtoT)—an autoregressive LM architecture that re…