2026
Revisiting Anisotropy in Language Transformers: The Geometry of Learning Dynamics
ICML 2026poster
Since their introduction, Transformer architectures have dominated Natural Language Processing (NLP). However, recent research has highlighted an inherent anisotropy phenomenon in these models, presenting a significant challenge to their geometric interpretation. Previous theoretical studies on this…