Shane Legg — accepted AI-conference papers

2024

Position: Levels of AGI for Operationalizing Progress on the Path to AGI

Meredith Ringel Morris, Jascha Sohl-Dickstein, Noah Fiedel, Tris Warkentin, Allan Dafoe, Aleksandra Faust +2

ICML 2024spotlight

We propose a framework for classifying the capabilities and behavior of Artificial General Intelligence (AGI) models and their precursors. This framework introduces levels of AGI performance, generality, and autonomy, providing a common language to compare models, assess risks, and measure progress…

Cited by 115 Source ↗PDF ↗Scholar ↗

2023

Neural Networks and the Chomsky Hierarchy

Gregoire Deletang, Anian Ruoss, Jordi Grau-Moya, Tim Genewein, Li Kevin Wenliang, Elliot Catt +5

ICLR 2023top-25%

Reliable generalization lies at the heart of safe ML and AI. However, understanding when and how neural networks generalize remains one of the most important unsolved problems in the field. In this work, we conduct an extensive empirical study (20'910 models, 15 tasks) to investigate whether insight…

Cited by 155 Source ↗PDF ↗code ↗Scholar ↗

2023

Randomized Positional Encodings Boost Length Generalization of Transformers

Anian Ruoss, Grégoire Delétang, Tim Genewein, Jordi Grau-Moya, Róbert Csordás, Mehdi Bennani +2

ACL 2023short

Transformers have impressive generalization capabilities on tasks with a fixed context length. However, they fail to generalize to sequences of arbitrary length, even for seemingly simple tasks such as duplicating a string. Moreover, simply training on longer sequences is inefficient due to the quad…

Cited by 97 Source ↗PDF ↗code ↗Scholar ↗

2021

Agent Incentives: A Causal Perspective

Tom Everitt, Ryan Carey, Eric D. Langlois, Pedro A. Ortega, Shane Legg

AAAI 2021technical

We present a framework for analysing agent incentives using causal influence diagrams. We establish that a well-known criterion for value of information is complete. We propose a new graphical criterion for value of control, establishing its soundness and completeness. We also introduce two new conc…

Cited by 64 Source ↗PDF ↗code ↗Scholar ↗

2021

Quantifying Differences in Reward Functions

Adam Gleave, Michael D Dennis, Shane Legg, Stuart Russell, Jan Leike

ICLR 2021spotlight

For many tasks, the reward function is inaccessible to introspection or too complex to be specified procedurally, and must instead be learned from user data. Prior work has evaluated learned reward functions by evaluating policies optimized for the learned reward. However, this method cannot disting…

Cited by 85 Source ↗PDF ↗code ↗Scholar ↗

2020

Avoiding Side Effects By Considering Future Tasks

Victoria Krakovna, Laurent Orseau, Richard Ngo, Miljan Martic, Shane Legg

NeurIPS 2020poster

Designing reward functions is difficult: the designer has to specify what to do (what it means to complete the task) as well as what not to do (side effects that should be avoided while completing the task). To alleviate the burden on the reward designer, we propose an algorithm to automatically gen…

Cited by 56 Source ↗PDF ↗code ↗Scholar ↗

2020

Learning Human Objectives by Evaluating Hypothetical Behavior

Siddharth Reddy, Anca Dragan, Sergey Levine, Shane Legg, Jan Leike

ICML 2020poster

We seek to align agent behavior with a user’s objectives in a reinforcement learning setting with unknown dynamics, an unknown reward function, and unknown unsafe states. The user knows the rewards and unsafe states, but querying the user is expensive. We propose an algorithm that safely and efficie…

Cited by 95 Source ↗PDF ↗Scholar ↗

2020

Meta-trained agents implement Bayes-optimal agents

Vladimir Mikulik, Grégoire Delétang, Tom McGrath, Tim Genewein, Miljan Martic, Shane Legg +1

NeurIPS 2020spotlight

Memory-based meta-learning is a powerful technique to build agents that adapt fast to any task within a target distribution. A previous theoretical study has argued that this remarkable performance is because the meta-training protocol incentivises agents to behave Bayes-optimally. We empirically in…

Cited by 49 Source ↗PDF ↗Scholar ↗

2020

Pitfalls of Learning a Reward Function Online

Stuart Armstrong, Jan Leike, Laurent Orseau, Shane Legg

IJCAI 2020poster

In some agent designs like inverse reinforcement learning an agent needs to learn its own reward function. Learning the reward function and optimising for it are typically two different processes, usually performed at different stages. We consider a continual (``one life'') learning approach where t…

Cited by 0 Source ↗PDF ↗Scholar ↗

2018

IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures

Lasse Espeholt, Hubert Soyer, Remi Munos, Karen Simonyan, Vlad Mnih, Tom Ward +6

ICML 2018oral

In this work we aim to solve a large collection of tasks using a single reinforcement learning agent with a single set of parameters. A key challenge is to handle the increased amount of data and extended training time. We have developed a new distributed agent IMPALA (Importance Weighted Actor-Lear…

Cited by 1853 Source ↗PDF ↗code ↗Scholar ↗

2018

Noisy Networks For Exploration

Meire Fortunato, Mohammad Gheshlaghi Azar, Bilal Piot, Jacob Menick, Matteo Hessel, Ian Osband +7

ICLR 2018poster

We introduce NoisyNet, a deep reinforcement learning agent with parametric noise added to its weights, and show that the induced stochasticity of the agent’s policy can be used to aid efficient exploration. The parameters of the noise are learned with gradient descent along with the remaining networ…

Cited by 1259 Source ↗PDF ↗Scholar ↗

2018

Reward learning from human preferences and demonstrations in Atari

Borja Ibarz, Jan Leike, Tobias Pohlen, Geoffrey Irving, Shane Legg, Dario Amodei

NeurIPS 2018poster

To solve complex real-world problems with reinforcement learning, we cannot rely on manually specified reward functions. Instead, we need humans to communicate an objective to the agent directly. In this work, we combine two approaches to this problem: learning from expert demonstrations and learnin…

Cited by 504 Source ↗PDF ↗Scholar ↗

2017

Deep Reinforcement Learning from Human Preferences

Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, Dario Amodei

NeurIPS 2017poster

For sophisticated reinforcement learning (RL) systems to interact usefully with real-world environments, we need to communicate complex goals to these systems. In this work, we explore goals defined in terms of (non-expert) human preferences between pairs of trajectory segments. Our approach separat…

Cited by 4197 Source ↗PDF ↗Scholar ↗