2025
Recurrent Self-Attention Dynamics: An Energy-Agnostic Perspective from Jacobians
NeurIPS 2025poster
The theoretical understanding of self-attention (SA) has been steadily progressing. A prominent line of work studies a class of SA layers that admit an energy function decreased by state updates. While it provides valuable insights into inherent biases in signal propagation, it often relies on ideal…