← Search

Benjamin Ellis

6 accepted papers

2025

OvercookedV2: Rethinking Overcooked for Zero-Shot Coordination

ICLR 2025poster

AI agents hold the potential to transform everyday life by helping humans achieve their goals. To do this successfully, agents need to be able to coordinate with novel partners without prior interaction, a setting known as zero-shot coordination (ZSC). Overcooked has become one of the most popular b…

2025

Simplifying Deep Temporal Difference Learning

ICLR 2025spotlight

$Q$-learning played a foundational role in the field reinforcement learning (RL). However, TD algorithms with off-policy data, such as $Q$-learning, or nonlinear function approximation like deep neural networks require several additional tricks to stabilise training, primarily a large replay buffer…

2024

Adam on Local Time: Addressing Nonstationarity in RL with Relative Adam Timesteps

NeurIPS 2024poster

In reinforcement learning (RL), it is common to apply techniques used broadly in machine learning such as neural network function approximators and momentum-based optimizers. However, such tools were largely developed for supervised learning rather than nonstationary RL, leading practitioners to ad…

Cited by 0SourcePDFScholar
2024

Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learning

ICML 2024spotlight

Benchmarks play a crucial role in the development and analysis of reinforcement learning (RL) algorithms. We identify that existing benchmarks used for research into open-ended learning fall into one of two categories. Either they are too slow for meaningful research to be performed without enormous…

2024

JaxMARL: Multi-Agent RL Environments and Algorithms in JAX

NeurIPS 2024poster

Benchmarks are crucial in the development of machine learning algorithms, significantly influencing reinforcement learning (RL) research through the available environments. Traditionally, RL environments run on the CPU, which limits their scalability with the computational resources typically availa…

2023

SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learning

NeurIPS 2023poster

The availability of challenging benchmarks has played a key role in the recent progress of machine learning. In cooperative multi-agent reinforcement learning, the StarCraft Multi-Agent Challenge (SMAC) has become a popular testbed for centralised training with decentralised execution. However, aft…