AIConfPaper
← Search

Charline Le Lan

9 accepted papers

2024

Human Alignment of Large Language Models through Online Preference Optimisation

Daniele Calandriello, Zhaohan Daniel Guo, Remi Munos, Mark Rowland, Yunhao Tang, Bernardo Avila Pires +7
ICML 2024poster

Ensuring alignment of language model's outputs with human preferences is critical to guarantee a useful, safe, and pleasant user experience. Thus, human alignment has been extensively studied recently and several methods such as Reinforcement Learning from Human Feedback (RLHF), Direct Policy Optimi…

Cited by 40Source ↗PDF ↗Scholar ↗
2023

A Novel Stochastic Gradient Descent Algorithm for Learning Principal Subspaces

Charline Le Lan, Joshua Greaves, Jesse Farebrother, Mark Rowland, Fabian Pedregosa, Rishabh Agarwal +1
AISTATS 2023poster

Many machine learning problems encode their data as a matrix with a possibly very large number of rows and columns. In several applications like neuroscience, image compression or deep reinforcement learning, the principal subspace of such a matrix provides a useful, low-dimensional representation o…

Cited by 6Source ↗PDF ↗code ↗Scholar ↗
2023

Bootstrapped Representations in Reinforcement Learning

Charline Le Lan, Stephen Tu, Mark Rowland, Anna Harutyunyan, Rishabh Agarwal, Marc G Bellemare +1
ICML 2023poster

In reinforcement learning (RL), state representations are key to dealing with large or continuous state spaces. While one of the promises of deep learning algorithms is to automatically construct features well-tuned for the task they try to solve, such a representation might not emerge from end-to-e…

Cited by 8Source ↗PDF ↗Scholar ↗
2023

Proto-Value Networks: Scaling Representation Learning with Auxiliary Tasks

Jesse Farebrother, Joshua Greaves, Rishabh Agarwal, Charline Le Lan, Ross Goroshin, Pablo Samuel Castro +1
ICLR 2023poster

Auxiliary tasks improve the representations learned by deep reinforcement learning agents. Analytically, their effect is reasonably well-understood; in practice, how-ever, their primary use remains in support of a main learning objective, rather than as a method for learning representations. This is…

Cited by 12Source ↗PDF ↗code ↗Scholar ↗
2023

Understanding Self-Predictive Learning for Reinforcement Learning

Yunhao Tang, Zhaohan Daniel Guo, Pierre Harvey Richemond, Bernardo Avila Pires, Yash Chandak, Remi Munos +10
ICML 2023poster

We study the learning dynamics of self-predictive learning for reinforcement learning, a family of algorithms that learn representations by minimizing the prediction error of their own future latent representations. Despite its recent empirical success, such algorithms have an apparent defect: trivi…

Cited by 34Source ↗PDF ↗Scholar ↗
2022

On the Generalization of Representations in Reinforcement Learning

Charline Le Lan, Stephen Tu, Adam Oberman, Rishabh Agarwal, Marc G. Bellemare
AISTATS 2022poster

In reinforcement learning, state representations are used to tractably deal with large problem spaces. State representations serve both to approximate the value function with few parameters, but also to generalize to newly encountered states. Their features may be learned implicitly (as part of a ne…

Cited by 41Source ↗PDF ↗code ↗Scholar ↗
2021

LieTransformer: Equivariant Self-Attention for Lie Groups

Michael J Hutchinson, Charline Le Lan, Sheheryar Zaidi, Emilien Dupont, Yee Whye Teh, Hyunjik Kim
ICML 2021spotlight

Group equivariant neural networks are used as building blocks of group invariant neural networks, which have been shown to improve generalisation performance and data efficiency through principled parameter sharing. Such works have mostly focused on group equivariant convolutions, building on the re…

Cited by 71Source ↗PDF ↗code ↗Scholar ↗
2021

Metrics and Continuity in Reinforcement Learning

Charline Le Lan, Marc G. Bellemare, Pablo Samuel Castro
AAAI 2021technical

In most practical applications of reinforcement learning, it is untenable to maintain direct estimates for individual states; in continuous-state systems, it is impossible. Instead, researchers often leverage {em state similarity} (whether explicitly or implicitly) to build models that can generali…

Cited by 35Source ↗PDF ↗code ↗Scholar ↗
2019

Continuous Hierarchical Representations with Poincaré Variational Auto-Encoders

Emile Mathieu, Charline Le Lan, Chris J. Maddison, Ryota Tomioka, Yee Whye Teh
NeurIPS 2019poster

The Variational Auto-Encoder (VAE) is a popular method for learning a generative model and embeddings of the data. Many real datasets are hierarchically structured. However, traditional VAEs map data in a Euclidean latent space which cannot efficiently embed tree-like structures. Hyperbolic spaces…

Cited by 217Source ↗PDF ↗code ↗Scholar ↗