2026
When Softmax Fails at the Top: Extreme‑Value Corrections for InfoNCE
Hasan Sabri Melihcan Erol, Suat Evren, Oktay Ozel, Alexander Morgan, Jongha (Jon) Ryu, Lizhong Zheng
ICML 2026poster
Contrastive learning is often trained with the InfoNCE loss, which uses a softmax over similarities to make the positive pair score higher than many negatives. Beyond its connection to mutual information, this softmax link has a precise probabilistic meaning: it is the maximum likelihood objective o…