AIConfPaper
← Search

Vivien Cabannes

14 accepted papers

2025

$\mathbb{X}$-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs

Vlad Sobal, Mark Ibrahim, Randall Balestriero, Vivien Cabannes, Diane Bouchacourt, Pietro Astolfi +2
ICLR 2025poster

Learning good representations involves capturing the diverse ways in which data samples relate. Contrastive loss—an objective matching related samples—underlies methods from self-supervised to multimodal learning. Contrastive losses, however, can be viewed more broadly as modifying a similarity grap…

Cited by 0Source ↗PDF ↗Scholar ↗
2025

Asymmetric REINFORCE for off-Policy Reinforcement Learning: Balancing positive and negative rewards

Charles Arnal, Gaëtan Narozniak, Vivien Cabannes, Yunhao Tang, Julia Kempe, Remi Munos
NeurIPS 2025poster

Reinforcement learning (RL) is increasingly used to align large language models (LLMs). Off-policy methods offer greater implementation simplicity and data efficiency than on-policy techniques, but often result in suboptimal performance. In this work, we study the intermediate range of algorithms be…

Cited by 0Source ↗Scholar ↗
2024

Iteration Head: A Mechanistic Study of Chain-of-Thought

Vivien Cabannes, Charles Arnal, Wassim Bouaziz, Xingyu Alice Yang, Francois Charton, Julia Kempe
NeurIPS 2024poster

Chain-of-Thought (CoT) reasoning is known to improve Large Language Models both empirically and in terms of theoretical approximation power. However, our understanding of the inner workings and conditions of apparition of CoT capabilities remains limited. This paper helps fill this gap by demonstrat…

Cited by 7Source ↗PDF ↗code ↗Scholar ↗
2023

Active Self-Supervised Learning: A Few Low-Cost Relationships Are All You Need

Vivien Cabannes, Leon Bottou, Yann Lecun, Randall Balestriero
ICCV 2023poster

Self-Supervised Learning (SSL) has emerged as the solution of choice to learn transferable representations from unlabeled data. However, SSL requires to build samples that are known to be semantically akin, i.e. positive views. Requiring such knowledge is the main limitation of SSL and is often tac…

Cited by 18PDF ↗code ↗Scholar ↗
2023

Birth of a Transformer: A Memory Viewpoint

Alberto Bietti, Vivien Cabannes, Diane Bouchacourt, Herve Jegou, Leon Bottou
NeurIPS 2023spotlight

Large language models based on transformers have achieved great empirical successes. However, as they are deployed more widely, there is a growing need to better understand their internal mechanisms in order to make them more reliable. These models appear to store vast amounts of knowledge from thei…

Cited by 86Source ↗PDF ↗Scholar ↗
2023

The SSL Interplay: Augmentations, Inductive Bias, and Generalization

Vivien Cabannes, Bobak Kiani, Randall Balestriero, Yann LeCun, Alberto Bietti
ICML 2023poster

Self-supervised learning (SSL) has emerged as a powerful framework to learn representations from raw data without supervision. Yet in practice, engineers face issues such as instability in tuning optimizers and collapse of representations during training. Such challenges motivate the need for a theo…

Cited by 41Source ↗PDF ↗Scholar ↗
2022

Active Labeling: Streaming Stochastic Gradients

Vivien Cabannes, Francis Bach, Vianney Perchet, Alessandro Rudi
NeurIPS 2022accept

The workhorse of machine learning is stochastic gradient descent. To access stochastic gradients, it is common to consider iteratively input/output pairs of a training dataset. Interestingly, it appears that one does not need full supervision to access stochastic gradients, which is the main motivat…

Cited by 2Source ↗PDF ↗code ↗Scholar ↗
2021

Overcoming the curse of dimensionality with Laplacian regularization in semi-supervised learning

Vivien Cabannes, Loucas Pillaud-Vivien, Francis Bach, Alessandro Rudi
NeurIPS 2021poster

As annotations of data can be scarce in large-scale practical problems, leveraging unlabelled examples is one of the most important aspects of machine learning. This is the aim of semi-supervised learning. To benefit from the access to unlabelled data, it is natural to diffuse smoothly knowledge of…

Cited by 29Source ↗PDF ↗code ↗Scholar ↗