Probabilistic Contrastive Test-Time Adaptation

Abstract

Test-time adaptation (TTA) enhances generalization against out-of-distribution data during inference. Recent advances in TTA leverage some techniques such as contrastive learning and entropy minimization to improve the discriminability and robustness of models in target domains. However, existing methods often overlook simultaneous distribution shifts of sample and label, such as long-tail distributions, and contrastive learning approaches may require substantial storage for sample pairs. In this paper, we propose a novel Probabilistic Contrastive Test-time Adaptation (PCTA) method based on Expectation Maximization (EM), which is used to estimate the von Mises Fisher (vMF) distribution of test samples to capture both sample distribution and class proportions. The estimated distributions are used for probabilistic contrastive learning to adapt feature representations and optimize classification through class-weighted entropy minimization. Experimental results show that PCTA significantly enhances the performance across various distribution shifts and outperforms state-of-the-art methods in different scenarios involving both sample and label shifts. Code is available at https://github.com/youlj109/PCTA.