Max Gonzalez Saez-Diez

Unifying Specialized Visual Encoders for Video Language Models

Recent advances in vision backbones have yielded powerful and diverse visual and video encoders. Yet, current Video Large Language Models encode visual inputs using an encoder from a single backbone family, limiting the amount and type of visual information they can process. We propose MERV, a Multi…