← Search

Michel Ma

3 accepted papers

2024

Bridging State and History Representations: Understanding Self-Predictive RL

ICLR 2024poster

Representations are at the core of all deep reinforcement learning (RL) methods for both Markov decision processes (MDPs) and partially observable Markov decision processes (POMDPs). Many representation learning methods and theoretical frameworks have been developed to understand what constitutes an…

2024

Do Transformer World Models Give Better Policy Gradients?

ICML 2024poster

A natural approach for reinforcement learning is to predict future rewards by unrolling a neural network world model, and to backpropagate through the resulting computational graph to learn a control policy. However, this method often becomes impractical for long horizons, since typical world models…

Cited by 3SourcePDFScholar
2023

When Do Transformers Shine in RL? Decoupling Memory from Credit Assignment

NeurIPS 2023oral

Reinforcement learning (RL) algorithms face two distinct challenges: learning effective representations of past and present observations, and determining how actions influence future returns. Both challenges involve modeling long-term dependencies. The Transformer architecture has been very successf…