Olivier J. Henaff — accepted AI-conference papers

2025

Active Data Curation Effectively Distills Large-Scale Multimodal Models

Vishaal Udandarao, Nikhil Parthasarathy, Muhammad Ferjad Naeem, Talfan Evans, Samuel Albanie, Federico Tombari +3

CVPR 2025poster

Knowledge distillation (KD) is the de facto standard for compressing large-scale models into smaller ones. Prior works have explored ever more complex KD strategies involving different objective functions, teacher-ensembles, and weight inheritance. In this work we explore an alternative, yet simple…

Cited by 6 Source ↗PDF ↗Scholar ↗

2025

Context-Aware Multimodal Pretraining

Karsten Roth, Zeynep Akata, Dima Damen, Ivana Balazevic, Olivier J. Henaff

CVPR 2025highlight

Large-scale multimodal representation learning successfully optimizes for zero-shot transfer at test time. Yet the standard pretraining paradigm (contrastive learning on large amounts of image-text data) does not explicitly encourage representations to support few-shot adaptation. In this work, we p…

Cited by 2 Source ↗PDF ↗code ↗Scholar ↗

2024

A Practitioner's Guide to Real-World Continual Multimodal Pretraining

Vishaal Udandarao, Karsten Roth, Sebastian Dziadzio, Ameya Prabhu, Mehdi Cherti, Oriol Vinyals +4

NeurIPS 2024poster

Multimodal foundation models serve numerous applications at the intersection of vision and language. Still, despite being pretrained on extensive data, they become outdated over time. To keep models updated, research into continual pretraining mainly explores scenarios with either (1) infrequent, in…

Cited by 0 Source ↗PDF ↗code ↗Scholar ↗

2024

Data curation via joint example selection further accelerates multimodal learning

Talfan Evans, Nikhil Parthasarathy, Hamza Merzic, Olivier J Henaff

NeurIPS 2024spotlight

Data curation is an essential component of large-scale pretraining. In this work, we demonstrate that jointly prioritizing batches of data is more effective for learning than selecting examples independently. Multimodal contrastive objectives expose the dependencies between data and thus naturally y…

Cited by 18 Source ↗PDF ↗Scholar ↗

2024

Fantastic Gains and Where to Find Them: On the Existence and Prospect of General Knowledge Transfer between Any Pretrained Model

Karsten Roth, Lukas Thede, A. Sophia Koepke, Oriol Vinyals, Olivier J Henaff, Zeynep Akata

ICLR 2024spotlight

Training deep networks requires various design decisions regarding for instance their architecture, data augmentation, or optimization. In this work, we find these training variations to result in networks learning unique feature sets from the data. Using public model libraries comprising thousands…

Cited by 11 Source ↗PDF ↗code ↗Scholar ↗

2024

Memory Consolidation Enables Long-Context Video Understanding

Ivana Balazevic, Yuge Shi, Pinelopi Papalampidi, Rahma Chaabouni, Skanda Koppula, Olivier J Henaff

ICML 2024spotlight

Most transformer-based video encoders are limited to short temporal contexts due to their quadratic complexity. While various attempts have been made to extend this context, this has often come at the cost of both conceptual and computational complexity. We propose to instead re-purpose existing pre…

Cited by 25 Source ↗PDF ↗Scholar ↗

2023

Self-supervised video pretraining yields robust and more human-aligned visual representations

Nikhil Parthasarathy, S. M. Ali Eslami, Joao Carreira, Olivier J Henaff

NeurIPS 2023poster

Humans learn powerful representations of objects and scenes by observing how they evolve over time. Yet, outside of specific tasks that require explicit temporal understanding, static image pretraining remains the dominant paradigm for learning visual foundation models. We question this mismatch, an…

Cited by 19 Source ↗PDF ↗Scholar ↗

2023

Towards In-context Scene Understanding

Ivana Balazevic, David Steiner, Nikhil Parthasarathy, Relja Arandjelovic, Olivier J Henaff

NeurIPS 2023spotlight

In-context learning––the ability to configure a model's behavior with different prompts––has revolutionized the field of natural language processing, alleviating the need for task-specific models and paving the way for generalist models capable of assisting with any query. Computer vision, in contra…

Cited by 38 Source ↗PDF ↗Scholar ↗

2022

Perceiver IO: A General Architecture for Structured Inputs & Outputs

Andrew Jaegle, Sebastian Borgeaud, Jean-Baptiste Alayrac, Carl Doersch, Catalin Ionescu, David Ding +9

ICLR 2022spotlight

A central goal of machine learning is the development of systems that can solve many problems in as many data domains as possible. Current architectures, however, cannot be applied beyond a small set of stereotyped settings, as they bake in domain & task assumptions or scale poorly to large inputs o…

Cited by 698 Source ↗PDF ↗code ↗Scholar ↗