AIConfPaper
← Search

Louis Béthune

20 accepted papers

2026

Completed Hyperparameter Transfer across Modules, Width, Depth, Batch and Duration

Bruno Kacper Mlodozeniec, Pierre Ablin, Louis Béthune, Dan Busbridge, Michal Klein, Jason Ramapuram +1
ICLR 2026poster

Hyperparameter tuning can dramatically impact training stability of large-scale models. Recent works on neural network parameterisations, such as μP, have shown that layer types and sizes should dictate how global hyperparameters should be rescaled in order to achieve efficient transfer across mode…

Cited by 0Source ↗Scholar ↗
2026

Learning Unmasking Policies for Diffusion Language Models

Metod Jazbec, Theo X. Olausson, Louis Béthune, Pierre Ablin, Michael Kirchhof, Joao Monteiro +3
ICML 2026oral

Diffusion (Large) Language Models (dLLMs) now match the downstream performance of their autoregressive counterparts on many tasks, while holding the promise of being more efficient during inference. One critical design aspect of dLLMs is the \textit{sampling procedure} that selects which tokens to u…

Cited by 0Source ↗Scholar ↗
2026

Optimal Splitting of Language Models from Mixtures to Specialized Domains

Skyler Seto, Pierre Ablin, Anastasiia Filippova, Jiayuan Ye, Louis Béthune, Angelos Katharopoulos +1
ICML 2026poster

Language models achieve impressive performance on a variety of knowledge, language, and reasoning tasks due to the scale and diversity of pretraining data available. The standard training recipe is a two-stage paradigm: pretraining first on the full corpus of data followed by specialization on a muc…

Cited by 0Source ↗Scholar ↗
2026

Removing Noise, not Finding Gold: Quality Filtering for Large-Scale Pretraining

Thiziri Nait Saada, Louis Béthune, Michal Klein, David Grangier, Marco Cuturi, Pierre Ablin
ICML 2026poster

Large-scale models are pretrained on massive web-crawled datasets containing documents of mixed quality, making data filtering essential. A popular method is Classifier-based Quality Filtering (CQF), which trains a binary classifier to distinguish between pretraining data and a small, high-quality s…

Cited by 0Source ↗Scholar ↗
2025

Composition and Control with Distilled Energy Diffusion Models and Sequential Monte Carlo

James Thornton, Louis Béthune, Ruixiang ZHANG, Arwen Bradley, Preetum Nakkiran, Shuangfei Zhai
AISTATS 2025poster

Diffusion models may be formulated as a time-indexed sequence of energy-based models, where the score corresponds to the negative gradient of an energy function. As opposed to learning the score directly, an energy parameterization is attractive as the energy itself can be used to control generation…

Cited by 25Source ↗Scholar ↗
2025

Deep Sturm–Liouville: From Sample-Based to 1D Regularization with Learnable Orthogonal Basis Functions

David Vigouroux, Joseba Dalmau, Louis Béthune, Victor Boutin
ICML 2025poster

Although Artificial Neural Networks (ANNs) have achieved remarkable success across various tasks, they still suffer from limited generalization. We hypothesize that this limitation arises from the traditional sample-based (0--dimensionnal) regularization used in ANNs. To overcome this, we introduce…

Cited by 0Source ↗PDF ↗Scholar ↗
2025

Follow the Energy, Find the Path: Riemannian Metrics from Energy-Based Models

Louis Béthune, David Vigouroux, Yilun Du, Rufin VanRullen, Thomas Serre, Victor Boutin
NeurIPS 2025poster

What is the shortest path between two data points lying in a high-dimensional space? While the answer is trivial in Euclidean geometry, it becomes significantly more complex when the data lies on a curved manifold—requiring a Riemannian metric to describe the space's local curvature. Estimating such…

Cited by 0Source ↗Scholar ↗
2025

Multimodal Autoregressive Pre-training of Large Vision Encoders

Enrico Fini, Mustafa Shukor, Xiujun Li, Philipp Dufter, Michal Klein, David Haldimann +10
CVPR 2025highlight

We introduce a novel method for pre-training of large-scale vision encoders. Building on recent advancements in autoregressive pre-training of vision models, we extend this framework to a multimodal setting, i.e., images and text. In this paper, we present AIMV2, a family of generalist vision encode…

Cited by 25Source ↗PDF ↗code ↗Scholar ↗
2025

Sample and Map from a Single Convex Potential: Generation using Conjugate Moment Measures

Nina Vesseron, Louis Béthune, marco cuturi
NeurIPS 2025poster

The canonical approach in generative modeling is to split model fitting into two blocks: define first how to sample noise (e.g. Gaussian) and choose next what to do with it (e.g. using a single map or flows). We explore in this work an alternative route that ties sampling and mapping. We find inspir…

Cited by 0Source ↗Scholar ↗
2025

Scaling Laws for Forgetting during Finetuning with Pretraining Data Injection

Louis Béthune, David Grangier, Dan Busbridge, Eleonora Gualdoni, marco cuturi, Pierre Ablin
ICML 2025poster

A widespread strategy to obtain a language model that performs well on a target domain is to finetune a pretrained model to perform unsupervised next-token prediction on data from that target domain. Finetuning presents two challenges: \textit{(i)} if the amount of target data is limited, as in most…

Cited by 1Source ↗PDF ↗Scholar ↗
2025

Scaling Laws for Optimal Data Mixtures

Mustafa Shukor, Louis Béthune, Dan Busbridge, David Grangier, Enrico Fini, Alaaeldin El-Nouby +1
NeurIPS 2025poster

Large foundation models are typically trained on data from multiple domains, with the data mixture--the proportion of each domain used--playing a critical role in model performance. The standard approach to selecting this mixture relies on trial and error, which becomes impractical for large-scale p…

Cited by 0Source ↗Scholar ↗
2025

Shielded Diffusion: Generating Novel and Diverse Images using Sparse Repellency

Michael Kirchhof, James Thornton, Louis Béthune, Pierre Ablin, Eugene Ndiaye, marco cuturi
ICML 2025poster

The adoption of text-to-image diffusion models raises concerns over reliability, drawing scrutiny under the lens of various metrics like calibration, fairness, or compute efficiency. We focus in this work on two issues that arise when deploying these models: a lack of diversity when prompting images…

Cited by 0Source ↗PDF ↗Scholar ↗
2024

DP-SGD Without Clipping: The Lipschitz Neural Network Way

Louis Béthune, Thomas Massena, Thibaut Boissin, Aurélien Bellet, Franck Mamalet, Yannick Prudent +3
ICLR 2024poster

State-of-the-art approaches for training Differentially Private (DP) Deep Neural Networks (DNN) face difficulties to estimate tight bounds on the sensitivity of the network's layers, and instead rely on a process of per-sample gradient clipping. This clipping process not only biases the direction of…

Cited by 9Source ↗PDF ↗code ↗Scholar ↗
2024

Understanding Visual Feature Reliance through the Lens of Complexity

Thomas FEL, Louis Béthune, Andrew Kyle Lampinen, Thomas Serre, Katherine Hermann
NeurIPS 2024poster

Recent studies suggest that deep learning models' inductive bias towards favoring simpler features may be an origin of shortcut learning. Yet, there has been limited focus on understanding the complexities of the myriad features that models learn. In this work, we introduce a new metric for quantify…

Cited by 5Source ↗PDF ↗Scholar ↗
2023

A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimation

Thomas FEL, Victor Boutin, Louis Béthune, Remi Cadene, Mazda Moayeri, Léo Andéol +2
NeurIPS 2023spotlight

In recent years, concept-based approaches have emerged as some of the most promising explainability methods to help us interpret the decisions of Artificial Neural Networks (ANNs). These methods seek to discover intelligible visual ``concepts'' buried within the complex patterns of ANN activations i…

Cited by 56Source ↗PDF ↗Scholar ↗
2023

CRAFT: Concept Recursive Activation FacTorization for Explainability

Thomas Fel, Agustin Picard, Louis Béthune, Thibaut Boissin, David Vigouroux, Julien Colin +2
CVPR 2023poster

Attribution methods are a popular class of explainability methods that use heatmaps to depict the most important areas of an image that drive a model decision. Nevertheless, recent work has shown that these methods have limited utility in practice, presumably because they only highlight the most sal…

Cited by 124Source ↗PDF ↗code ↗Scholar ↗
2023

Gaussian Processes on Distributions based on Regularized Optimal Transport

François Bachoc, Louis Béthune, Alberto Gonzalez-Sanz, Jean-Michel Loubes
AISTATS 2023poster

We present a novel kernel over the space of probability measures based on the dual formulation of optimal regularized transport. We propose an Hilbertian embedding of the space of probabilities using their Sinkhorn potentials, which are solutions of the dual entropic relaxed optimal transport betwee…

Cited by 15Source ↗PDF ↗Scholar ↗
2023

On the explainable properties of 1-Lipschitz Neural Networks: An Optimal Transport Perspective

Mathieu Serrurier, Franck Mamalet, Thomas FEL, Louis Béthune, Thibaut Boissin
NeurIPS 2023poster

Input gradients have a pivotal role in a variety of applications, including adversarial attack algorithms for evaluating model robustness, explainable AI techniques for generating saliency maps, and counterfactual explanations. However, saliency maps generated by traditional neural networks are ofte…

Cited by 10Source ↗PDF ↗Scholar ↗
2023

Robust One-Class Classification with Signed Distance Function using 1-Lipschitz Neural Networks

Louis Béthune, Paul Novello, Guillaume Coiffier, Thibaut Boissin, Mathieu Serrurier, Quentin VINCENOT +1
ICML 2023poster

We propose a new method, dubbed One Class Signed Distance Function (OCSDF), to perform One Class Classification (OCC) by provably learning the Signed Distance Function (SDF) to the boundary of the support of any distribution. The distance to the support can be interpreted as a normality score, and i…

Cited by 13Source ↗PDF ↗code ↗Scholar ↗
2022

Pay attention to your loss : understanding misconceptions about Lipschitz neural networks

Louis Béthune, Thibaut Boissin, Mathieu Serrurier, Franck Mamalet, Corentin Friedrich, Alberto Gonzalez Sanz
NeurIPS 2022accept

Lipschitz constrained networks have gathered considerable attention in the deep learning community, with usages ranging from Wasserstein distance estimation to the training of certifiably robust classifiers. However they remain commonly considered as less accurate, and their properties in learning a…

Cited by 47Source ↗PDF ↗Scholar ↗