AIConfPaper
← Search

Clare Lyle

24 accepted papers

2026

Position: Agentic AI systems should be making Bayes-consistent decisions

Theodore Papamarkou, Pierre Alquier, Matthias Bauer, Wray Buntine, Andrew Davison, Gintare Karolina Dziugaite +24
ICML 2026poster

LLMs excel at predictive tasks and complex reasoning tasks, but many high-value deployments rely on decisions under uncertainty, for example, which tool to call, which expert to consult, or how many resources to invest. While the usefulness and feasibility of Bayesian approaches remain unclear for L…

Cited by 0Source ↗Scholar ↗
2025

A Unifying Framework for Action-Conditional Self-Predictive Reinforcement Learning

Khimya Khetarpal, Zhaohan Daniel Guo, Bernardo Avila Pires, Yunhao Tang, Clare Lyle, Mark Rowland +4
AISTATS 2025poster

Learning a good representation is a crucial challenge for reinforcement learning (RL) agents. Self-predictive algorithms jointly learn a latent representation and dynamics model by bootstrapping from future latent representations (BYOL). Recent work has developed theoretical insights into these algo…

Cited by 0Source ↗Scholar ↗
2025

Plasticity as the Mirror of Empowerment

David Abel, Michael Bowling, Andre Barreto, Will Dabney, Shi Dong, Steven Stenberg Hansen +10
NeurIPS 2025spotlight

Agents are minimally entities that are influenced by their past observations and act to influence future observations. This latter capacity is captured by empowerment, which has served as a vital framing concept across artificial intelligence and cognitive science. This former capacity, however, is…

Cited by 0Source ↗Scholar ↗
2024

Mixtures of Experts Unlock Parameter Scaling for Deep RL

Johan Samir Obando Ceron, Ghada Sokar, Timon Willi, Clare Lyle, Jesse Farebrother, Jakob Nicolaus Foerster +3
ICML 2024spotlight

The recent rapid progress in (self) supervised learning models is in large part predicted by empirical scaling laws: a model's performance scales proportionally to its size. Analogous scaling laws remain elusive for reinforcement learning domains, however, where increasing the parameter count of a m…

Cited by 33Source ↗PDF ↗code ↗Scholar ↗
2024

Near-Minimax-Optimal Distributional Reinforcement Learning with a Generative Model

Mark Rowland, Li Kevin Wenliang, Remi Munos, Clare Lyle, Yunhao Tang, Will Dabney
NeurIPS 2024poster

We propose a new algorithm for model-based distributional reinforcement learning (RL), and prove that it is minimax-optimal for approximating return distributions in the generative model regime (up to logarithmic factors), the first result of this kind for any distributional RL algorithm. Our analys…

Cited by 3Source ↗PDF ↗Scholar ↗
2024

Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Reset

Alexandre Galashov, Michalis Titsias, András György, Clare Lyle, Razvan Pascanu, Yee Whye Teh +1
NeurIPS 2024poster

Neural networks are most often trained under the assumption that data come from a stationary distribution. However, settings in which this assumption is violated are of increasing importance; examples include supervised learning with distributional shifts, reinforcement learning, continual learning…

Cited by 4Source ↗PDF ↗Scholar ↗
2024

Normalization and effective learning rates in reinforcement learning

Clare Lyle, Zeyu Zheng, Khimya Khetarpal, James Martens, Hado van Hasselt, Razvan Pascanu +1
NeurIPS 2024poster

Normalization layers have recently experienced a renaissance in the deep reinforcement learning and continual learning literature, with several works highlighting diverse benefits such as improving loss landscape conditioning and combatting overestimation bias. However, normalization brings with it…

Cited by 6Source ↗PDF ↗Scholar ↗
2024

Slow and Steady Wins the Race: Maintaining Plasticity with Hare and Tortoise Networks

Hojoon Lee, Hyeonseo Cho, Hyunseung Kim, Donghu Kim, Dugki Min, Jaegul Choo +1
ICML 2024poster

This study investigates the loss of generalization ability in neural networks, revisiting warm-starting experiments from Ash & Adams. Our empirical analysis reveals that common methods designed to enhance plasticity by maintaining trainability provide limited benefits to generalization. While reinit…

Cited by 13Source ↗PDF ↗code ↗Scholar ↗
2023

Deep Reinforcement Learning with Plasticity Injection

Evgenii Nikishin, Junhyuk Oh, Georg Ostrovski, Clare Lyle, Razvan Pascanu, Will Dabney +1
NeurIPS 2023spotlight

A growing body of evidence suggests that neural networks employed in deep reinforcement learning (RL) gradually lose their plasticity, the ability to learn from new data; however, the analysis and mitigation of this phenomenon is hampered by the complex relationship between plasticity, exploration,…

Cited by 53Source ↗PDF ↗Scholar ↗
2023

DiscoBAX - Discovery of optimal intervention sets in genomic experiment design

Clare Lyle, Arash Mehrjou, Pascal Notin, Andrew Jesson, Stefan Bauer, Yarin Gal +1
ICML 2023poster

The discovery of therapeutics to treat genetically-driven pathologies relies on identifying genes involved in the underlying disease mechanism. Existing approaches search over the billions of potential interventions to maximize the expected influence on the target phenotype. However, to reduce the r…

Cited by 12Source ↗PDF ↗code ↗Scholar ↗
2023

The Statistical Benefits of Quantile Temporal-Difference Learning for Value Estimation

Mark Rowland, Yunhao Tang, Clare Lyle, Remi Munos, Marc G Bellemare, Will Dabney
ICML 2023poster

We study the problem of temporal-difference-based policy evaluation in reinforcement learning. In particular, we analyse the use of a distributional reinforcement learning algorithm, quantile temporal-difference learning (QTD), for this task. We reach the surprising conclusion that even if a practit…

Cited by 11Source ↗PDF ↗Scholar ↗
2023

Understanding Plasticity in Neural Networks

Clare Lyle, Zeyu Zheng, Evgenii Nikishin, Bernardo Avila Pires, Razvan Pascanu, Will Dabney
ICML 2023oral

Plasticity, the ability of a neural network to quickly change its predictions in response to new information, is essential for the adaptability and robustness of deep reinforcement learning systems. Deep neural networks are known to lose plasticity over the course of training even in relatively simp…

Cited by 114Source ↗PDF ↗Scholar ↗
2023

Understanding Self-Predictive Learning for Reinforcement Learning

Yunhao Tang, Zhaohan Daniel Guo, Pierre Harvey Richemond, Bernardo Avila Pires, Yash Chandak, Remi Munos +10
ICML 2023poster

We study the learning dynamics of self-predictive learning for reinforcement learning, a family of algorithms that learn representations by minimizing the prediction error of their own future latent representations. Despite its recent empirical success, such algorithms have an apparent defect: trivi…

Cited by 34Source ↗PDF ↗Scholar ↗
2022

Learning Dynamics and Generalization in Deep Reinforcement Learning

Clare Lyle, Mark Rowland, Will Dabney, Marta Kwiatkowska, Yarin Gal
ICML 2022spotlight

Solving a reinforcement learning (RL) problem poses two competing challenges: fitting a potentially discontinuous value function, and generalizing well to new observations. In this paper, we analyze the learning dynamics of temporal difference algorithms to gain novel insight into the tension betwee…

Cited by 39Source ↗PDF ↗Scholar ↗
2021

On the Effect of Auxiliary Tasks on Representation Dynamics

Clare Lyle, Mark Rowland, Georg Ostrovski, Will Dabney
AISTATS 2021poster

While auxiliary tasks play a key role in shaping the representations learnt by reinforcement learning agents, much is still unknown about the mechanisms through which this is achieved. This work develops our understanding of the relationship between auxiliary tasks, environment structure, and repres…

Cited by 84Source ↗PDF ↗Scholar ↗
2021

Provable Guarantees on the Robustness of Decision Rules to Causal Interventions

Benjie Wang, Clare Lyle, Marta Kwiatkowska
IJCAI 2021poster

Robustness of decision rules to shifts in the data-generating process is crucial to the successful deployment of decision-making systems. Such shifts can be viewed as interventions on a causal graph, which capture (possibly hypothetical) changes in the data-generating process, whether due to natural…

Cited by 7Source ↗PDF ↗code ↗Scholar ↗
2021

PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learning

Angelos Filos, Clare Lyle, Yarin Gal, Sergey Levine, Natasha Jaques, Gregory Farquhar
ICML 2021oral

We study reinforcement learning (RL) with no-reward demonstrations, a setting in which an RL agent has access to additional data from the interaction of other agents with the same environment. However, it has no access to the rewards or goals of these agents, and their objectives and levels of exper…

Cited by 40Source ↗PDF ↗code ↗Scholar ↗
2021

Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning

Jannik Kossen, Neil Band, Clare Lyle, Aidan Gomez, Tom Rainforth, Yarin Gal
NeurIPS 2021poster

We challenge a common assumption underlying most supervised deep learning: that a model makes a prediction depending only on its parameters and the features of a single input. To this end, we introduce a general-purpose deep learning architecture that takes as input the entire dataset instead of pro…

Cited by 164Source ↗PDF ↗code ↗Scholar ↗
2021

Speedy Performance Estimation for Neural Architecture Search

Binxin Ru, Clare Lyle, Lisa Schut, Miroslav Fil, Mark van der Wilk, Yarin Gal
NeurIPS 2021spotlight

Reliable yet efficient evaluation of generalisation performance of a proposed architecture is crucial to the success of neural architecture search (NAS). Traditional approaches face a variety of limitations: training each architecture to completion is prohibitively expensive, early stopped validatio…

Cited by 36Source ↗PDF ↗code ↗Scholar ↗
2020

A Bayesian Perspective on Training Speed and Model Selection

Clare Lyle, Lisa Schut, Robin Ru, Yarin Gal, Mark van der Wilk
NeurIPS 2020poster

We take a Bayesian perspective to illustrate a connection between training speed and the marginal likelihood in linear models. This provides two major insights: first, that a measure of a model's training speed can be used to estimate its marginal likelihood. Second, that this measure, under certain…

Cited by 35Source ↗PDF ↗Scholar ↗
2020

Invariant Causal Prediction for Block MDPs

Amy Zhang, Clare Lyle, Shagun Sodhani, Angelos Filos, Marta Kwiatkowska, Joelle Pineau +2
ICML 2020poster

Generalization across environments is critical to the successful application of reinforcement learning (RL) algorithms to real-world challenges. In this work we propose a method for learning state abstractions which generalize to novel observation distributions in the multi-environment RL setting. W…

Cited by 169Source ↗PDF ↗code ↗Scholar ↗
2019

A Geometric Perspective on Optimal Representations for Reinforcement Learning

Marc Bellemare, Will Dabney, Robert Dadashi, Adrien Ali Taiga, Pablo Samuel Castro, Nicolas Le Roux +3
NeurIPS 2019poster

We propose a new perspective on representation learning in reinforcement learning based on geometric properties of the space of value functions. From there, we provide formal evidence regarding the usefulness of value functions as auxiliary tasks in reinforcement learning. Our formulation considers…

Cited by 108Source ↗PDF ↗Scholar ↗