Taming "data-hungry" reinforcement learning? Stability in continuous state-action spaces

Yaqi Duan; Martin J Wainwright

NeurIPS 2024poster4 citations

Taming "data-hungry" reinforcement learning? Stability in continuous state-action spaces

Abstract

We introduce a novel framework for analyzing reinforcement learning (RL) in continuous state-action spaces, and use it to prove fast rates of convergence in both off-line and on-line settings. Our analysis highlights two key stability properties, relating to how changes in value functions and/or policies affect the Bellman operator and occupation measures. We argue that these properties are satisfied in many continuous state-action Markov decision processes. Our analysis also offers fresh perspectives on the roles of pessimism and optimism in off-line and on-line RL.

reinforcement learningcontinuous controlstability analysis

BibTeX

@inproceedings{
duan2024taming,
title={Taming ''data-hungry'' reinforcement learning? Stability in continuous state-action spaces},
author={Yaqi Duan and Martin J Wainwright},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=CbHz30KeA4}
}