MASS: Overcoming Language Bias in Image-Text Matching
Jiwan Chung, Seungwon Lim, Sangkyu Lee, Youngjae Yu
Abstract
Pretrained visual-language models have made significant advancements in multimodal tasks, including image-text retrieval. However, a major challenge in image-text matching lies in language bias, where models predominantly rely on language priors and neglect to adequately consider the visual content. We thus present Multimodal ASsociation Score (MASS), a framework that reduces the reliance on language priors for better visual accuracy in image-text matching problems. It can be seamlessly incorporated into existing visual-language models without necessitating additional training. Our experiments have shown that \modelname effectively lessens language bias without losing an understanding of linguistic compositionality. Overall, MASS offers a promising solution for enhancing image-text matching performance in visual-language models.
BibTeX
@article{Chung_Lim_Lee_Yu_2025, title={MASS: Overcoming Language Bias in Image-Text Matching}, volume={39}, url={https://ojs.aaai.org/index.php/AAAI/article/view/32262}, DOI={10.1609/aaai.v39i3.32262}, abstractNote={Pretrained visual-language models have made significant advancements in multimodal tasks, including image-text retrieval. However, a major challenge in image-text matching lies in language bias, where models predominantly rely on language priors and neglect to adequately consider the visual content. We thus present Multimodal ASsociation Score (MASS), a framework that reduces the reliance on language priors for better visual accuracy in image-text matching problems. It can be seamlessly incorporated into existing visual-language models without necessitating additional training. Our experiments have shown that \modelname effectively lessens language bias without losing an understanding of linguistic compositionality. Overall, MASS offers a promising solution for enhancing image-text matching performance in visual-language models.}, number={3}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Chung, Jiwan and Lim, Seungwon and Lee, Sangkyu and Yu, Youngjae}, year={2025}, month={Apr.}, pages={2591-2599} }