The Convergent Representation of Vision-Language Contrastive Learning: Geometry, Modality Gap and Shared Space Alignment
Multimodal contrastive learning (MCL) aims to embed data from two modalities in a shared embedding space. However, in practice, representations of images and text occupy completely separate regions of embedding space, a phenomenon called the modality gap. Moreover, experimental findings on how the s…