3 accepted papers
Cross-modal contrastive pre-training between natural language and other modalities, e.g., vision and audio, has demonstrated astonishing performance and effectiveness across a diverse variety of tasks and domains. In this paper, we investigate whether such natural language supervision can be used fo…
Video Language Models (VLMs) are crucial for generalizing across diverse tasks and using language cues to enhance learning. While transformer-based architectures have been the de facto in vision-language training, they face challenges like quadratic computational complexity, high GPU memory usage, a…
Anticipating future actions is a highly challenging task due to the diversity and scale of potential future actions; yet, additional and complementary information from different modalities help narrow down plausible action choices. Going beyond typical sources such as video and audio, we primarily e…