Xavi Gonzalvo

Equivalence of Context and Parameter Updates in Modern Transformer Blocks

Recent research has established that the impact of context in a vanilla transformer can be represented implicitly by forming a token-dependent, rank-1 patch to its MLP weights. This work extends that foundational theory to the diverse architectures of modern Large Language Models. We first demonstra…