Alexander Morgan

When Softmax Fails at the Top: Extreme‑Value Corrections for InfoNCE

Contrastive learning is often trained with the InfoNCE loss, which uses a softmax over similarities to make the positive pair score higher than many negatives. Beyond its connection to mutual information, this softmax link has a precise probabilistic meaning: it is the maximum likelihood objective o…