Murdock Aubry

Transformer Block Coupling and its Correlation with Generalization in LLMs

Large Language Models (LLMs) have made significant strides in natural language processing, and a precise understanding of the internal mechanisms driving their success is essential. In this work, we analyze the trajectories of token embeddings as they pass through transformer blocks, linearizing the…