AIConfPaper
← Search

Arthur Guez

15 accepted papers

2025

A Unifying Framework for Action-Conditional Self-Predictive Reinforcement Learning

Khimya Khetarpal, Zhaohan Daniel Guo, Bernardo Avila Pires, Yunhao Tang, Clare Lyle, Mark Rowland +4
AISTATS 2025poster

Learning a good representation is a crucial challenge for reinforcement learning (RL) agents. Self-predictive algorithms jointly learn a latent representation and dynamics model by bootstrapping from future latent representations (BYOL). Recent work has developed theoretical insights into these algo…

Cited by 0Source ↗Scholar ↗
2022

COptiDICE: Offline Constrained Reinforcement Learning via Stationary Distribution Correction Estimation

Jongmin Lee, Cosmin Paduraru, Daniel J Mankowitz, Nicolas Heess, Doina Precup, Kee-Eung Kim +1
ICLR 2022spotlight

We consider the offline constrained reinforcement learning (RL) problem, in which the agent aims to compute a policy that maximizes expected return while satisfying given cost constraints, learning only from a pre-collected dataset. This problem setting is appealing in many real-world scenarios, whe…

Cited by 66Source ↗PDF ↗code ↗Scholar ↗
2022

Large-Scale Retrieval for Reinforcement Learning

Peter Conway Humphreys, Arthur Guez, Olivier Tieleman, Laurent Sifre, Theophane Weber, Timothy P Lillicrap
NeurIPS 2022accept

Effective decision making involves flexibly relating past experiences and relevant contextual information to a novel situation. In deep reinforcement learning (RL), the dominant paradigm is for an agent to amortise information that helps decision-making into its network weights via gradient descent…

Cited by 28Source ↗PDF ↗Scholar ↗
2022

Policy improvement by planning with Gumbel

Ivo Danihelka, Arthur Guez, Julian Schrittwieser, David Silver
ICLR 2022spotlight

AlphaZero is a powerful reinforcement learning algorithm based on approximate policy iteration and tree search. However, AlphaZero can fail to improve its policy network, if not visiting all actions at the root of a search tree. To address this issue, we propose a policy improvement algorithm based…

Cited by 89Source ↗PDF ↗code ↗Scholar ↗
2022

Retrieval-Augmented Reinforcement Learning

Anirudh Goyal, Abram Friesen, Andrea Banino, Theophane Weber, Nan Rosemary Ke, Adrià Puigdomènech Badia +9
ICML 2022spotlight

Most deep reinforcement learning (RL) algorithms distill experience into parametric behavior policies or value functions via gradient updates. While effective, this approach has several disadvantages: (1) it is computationally expensive, (2) it can take many updates to integrate experiences into the…

Cited by 59Source ↗PDF ↗Scholar ↗
2021

Counterfactual Credit Assignment in Model-Free Reinforcement Learning

Thomas Mesnard, Theophane Weber, Fabio Viola, Shantanu Thakoor, Alaa Saade, Anna Harutyunyan +8
ICML 2021spotlight

Credit assignment in reinforcement learning is the problem of measuring an action’s influence on future rewards. In particular, this requires separating skill from luck, i.e. disentangling the effect of an action on rewards from that of external factors and subsequent actions. To achieve this, we ad…

Cited by 78Source ↗PDF ↗Scholar ↗
2021

Muesli: Combining Improvements in Policy Optimization

Matteo Hessel, Ivo Danihelka, Fabio Viola, Arthur Guez, Simon Schmitt, Laurent Sifre +3
ICML 2021spotlight

We propose a novel policy update that combines regularized policy optimization with model learning as an auxiliary loss. The update (henceforth Muesli) matches MuZero’s state-of-the-art performance on Atari. Notably, Muesli does so without using deep search: it acts directly with a policy network an…

Cited by 92Source ↗PDF ↗code ↗Scholar ↗
2021

On the role of planning in model-based deep reinforcement learning

Jessica B Hamrick, Abram L. Friesen, Feryal Behbahani, Arthur Guez, Fabio Viola, Sims Witherspoon +4
ICLR 2021poster

Model-based planning is often thought to be necessary for deep, careful reasoning and generalization in artificial agents. While recent successes of model-based reinforcement learning (MBRL) with deep function approximation have strengthened this hypothesis, the resulting diversity of model-based me…

Cited by 95Source ↗PDF ↗Scholar ↗
2020

Value-driven Hindsight Modelling

Arthur Guez, Fabio Viola, Theophane Weber, Lars Buesing, Steven Kapturowski, Doina Precup +2
NeurIPS 2020poster

Value estimation is a critical component of the reinforcement learning (RL) paradigm. The question of how to effectively learn value predictors from data is one of the major problems studied by the RL community, and different approaches exploit structure in the problem domain in different ways. Mod…

Cited by 22Source ↗PDF ↗Scholar ↗
2019

An Investigation of Model-Free Planning

Arthur Guez, Mehdi Mirza, Karol Gregor, Rishabh Kabra, Sebastien Racaniere, Theophane Weber +7
ICML 2019oral

The field of reinforcement learning (RL) is facing increasingly challenging domains with combinatorial complexity. For an RL agent to address these challenges, it is essential that it can plan effectively. Prior work has typically utilized an explicit model of the environment, combined with a specif…

Cited by 108Source ↗PDF ↗code ↗Scholar ↗
2019

Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Search

Lars Buesing, Theophane Weber, Yori Zwols, Nicolas Heess, Sebastien Racaniere, Arthur Guez +1
ICLR 2019poster

Learning policies on data synthesized by models can in principle quench the thirst of reinforcement learning algorithms for large amounts of real experience, which is often costly to acquire. However, simulating plausible experience de novo is a hard problem for many complex environments, often resu…

Cited by 166Source ↗PDF ↗Scholar ↗
2018

Learning to search with MCTSnets

Arthur Guez, Theophane Weber, Ioannis Antonoglou, Karen Simonyan, Oriol Vinyals, Daan Wierstra +2
ICML 2018oral

Planning problems are among the most important and well-studied problems in artificial intelligence. They are most typically solved by tree search algorithms that simulate ahead into the future, evaluate future states, and back-up those evaluations to the root of a search tree. Among these algorithm…

Cited by 107Source ↗PDF ↗Scholar ↗
2017

Imagination-Augmented Agents for Deep Reinforcement Learning

Sébastien Racanière, Theophane Weber, David Reichert, Lars Buesing, Arthur Guez, Danilo Jimenez Rezende +9
NeurIPS 2017oral

We introduce Imagination-Augmented Agents (I2As), a novel architecture for deep reinforcement learning combining model-free and model-based aspects. In contrast to most existing model-based reinforcement learning and planning methods, which prescribe how a model should be used to arrive at a polic…

Cited by 763Source ↗PDF ↗Scholar ↗
2017

The Predictron: End-To-End Learning and Planning

David Silver, Hado Hasselt, Matteo Hessel, Tom Schaul, Arthur Guez, Tim Harley +5
ICML 2017poster

One of the key challenges of artificial intelligence is to learn models that are effective in the context of planning. In this document we introduce the predictron architecture. The predictron consists of a fully abstract model, represented by a Markov reward process, that can be rolled forward mult…

Cited by 327Source ↗PDF ↗Scholar ↗
2016

Learning values across many orders of magnitude

Hado P van Hasselt, Arthur Guez, Matteo Hessel, Volodymyr Mnih, David Silver
NeurIPS 2016poster

Most learning algorithms are not invariant to the scale of the signal that is being approximated. We propose to adaptively normalize the targets used in the learning updates. This is important in value-based reinforcement learning, where the magnitude of appropriate value approximations can change…

Cited by 218Source ↗PDF ↗Scholar ↗