Mitigating the Modality Gap in Vision–Language Models with Fractal Spectral Geometry
Vision–language models such as CLIP embed images and text into a shared space, but still suffer from a modality gap, where image and text features cluster separately and nearest neighbors are dominated by same-modality rather than true cross-modal matches. Existing works alleviate the modality gap b…