Pavan Yeddanapudi

Contrastive Predictive Coding Done Right for Mutual Information Estimation

The InfoNCE objective, originally introduced for contrastive representation learning, has become a popular choice for mutual information (MI) estimation, despite its indirect connection to MI. In this paper, we demonstrate why InfoNCE should not be regarded as a valid MI estimator, and we introduce…