Victor Gomes

Mirasol3B: A Multimodal Autoregressive Model for Time-Aligned and Contextual Modalities

One of the main challenges of multimodal learning is the need to combine heterogeneous modalities (e.g. video audio text). For example video and audio are obtained at much higher rates than text and are roughly aligned in time. They are often not synchronized with text which comes as a global contex…