2025
A multiscale analysis of mean-field transformers in the moderate interaction regime
NeurIPS 2025oral
In this paper, we study the evolution of tokens through the depth of encoder-only transformer models at inference time by modeling them as a system of particles interacting in a mean-field way and studying the corresponding dynamics. More specifically, we consider this problem in the moderate intera…