ACL 2025short0 citations
Cross-Lingual Representation Alignment Through Contrastive Image-Caption Tuning
Nathaniel Krasner, Nicholas Lanuzo, Antonios Anastasopoulos
Abstract
Multilingual alignment of sentence representations has mostly required bitexts to bridge the gap between languages. We investigate whether visual information can bridge this gap instead. Image caption datasets are very easy to create without requiring multilingual expertise, so this offers a more efficient alternative for low-resource languages. We find that multilingual image-caption alignment can implicitly align the text representations between languages, languages unseen by the encoder in pretraining can be incorporated into this alignment post-hoc, and these aligned representations are usable for cross-lingual Natural Language Understanding (NLU) and bitext retrieval.
BibTeX
@inproceedings{krasner-etal-2025-cross,
title = "Cross-Lingual Representation Alignment Through Contrastive Image-Caption Tuning",
author = "Krasner, Nathaniel and
Lanuzo, Nicholas and
Anastasopoulos, Antonios",
editor = "Che, Wanxiang and
Nabende, Joyce and
Shutova, Ekaterina and
Pilehvar, Mohammad Taher",
booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)",
month = jul,
year = "2025",
address = "Vienna, Austria",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.acl-short.95/",
doi = "10.18653/v1/2025.acl-short.95",
pages = "1193--1199",
ISBN = "979-8-89176-252-7"
}