Adrian Javaloy

Mitigating Modality Collapse in Multimodal VAEs via Impartial Optimization

A number of variational autoencoders (VAEs) have recently emerged with the aim of modeling multimodal data, e.g., to jointly model images and their corresponding captions. Still, multimodal VAEs tend to focus solely on a subset of the modalities, e.g., by fitting the image while neglecting the capti…