2025
Continuity and Isolation Lead to Doubts or Dilemmas in Large Language Models
Hector Pasten, Felipe Urrutia, Hector Ignacio Jimenez Orellana, Cristian Buc Calderon, Cristobal Rojas, Alexander Kozachinskiy
NeurIPS 2025poster
Understanding how Transformers work and how they process information is key to the theoretical and empirical advancement of these machines. In this work, we demonstrate the existence of two phenomena in Transformers, namely _isolation_ and _continuity_. Both of these phenomena hinder Transformers t…