Bodie Rubacher

Attention Implements the Fisher Geometry of Exponential Families

Softmax attention is increasingly treated as a reusable inference primitive in transformers. Much prior theory covers Gaussian/linear models or assumes one shared quadratic query--key metric, which can fail for non-Gaussian exponential-family likelihoods with varying curvature. This risks overreadin…