Faris Chaudhry

The Geometry of Projection Heads: Conditioning, Invariance, and Collapse

We develop a geometric theory of projection heads in self-supervised learning by interpreting the head as a trainable metric on the backbone representation manifold. Our analysis reveals that head curvature and architectural asymmetry induce negative eigenvalues of the Hessian at collapsed equilibri…