ICML 2024poster6 citations
Distributional Bellman Operators over Mean Embeddings
Li Kevin Wenliang, Gregoire Deletang, Matthew Aitchison, Marcus Hutter, Anian Ruoss, Arthur Gretton, Mark Rowland
Abstract
We propose a novel algorithmic framework for distributional reinforcement learning, based on learning finite-dimensional mean embeddings of return distributions. The framework reveals a wide variety of new algorithms for dynamic programming and temporal-difference algorithms that rely on the sketch Bellman operator, which updates mean embeddings with simple linear-algebraic computations. We provide asymptotic convergence theory, and examine the empirical performance of the algorithms on a suite of tabular tasks. Further, we show that this approach can be straightforwardly combined with deep reinforcement learning.
BibTeX
@inproceedings{
wenliang2024distributional,
title={Distributional Bellman Operators over Mean Embeddings},
author={Li Kevin Wenliang and Gregoire Deletang and Matthew Aitchison and Marcus Hutter and Anian Ruoss and Arthur Gretton and Mark Rowland},
booktitle={Forty-first International Conference on Machine Learning},
year={2024},
url={https://openreview.net/forum?id=j2pLfsBm4J}
}