NeurIPS 2021poster17 citations
Average-Reward Learning and Planning with Options
Yi Wan, Abhishek Naik, Richard S. Sutton
Abstract
We extend the options framework for temporal abstraction in reinforcement learning from discounted Markov decision processes (MDPs) to average-reward MDPs. Our contributions include general convergent off-policy inter-option learning algorithms, intra-option algorithms for learning values and models, as well as sample-based planning variants of our learning algorithms. Our algorithms and convergence proofs extend those recently developed by Wan, Naik, and Sutton. We also extend the notion of option-interrupting behaviour from the discounted to the average-reward formulation. We show the efficacy of the proposed algorithms with experiments on a continuing version of the Four-Room domain.
average rewardoptionsreinforcement learning
BibTeX
@inproceedings{
wan2021averagereward,
title={Average-Reward Learning and Planning with Options},
author={Yi Wan and Abhishek Naik and Richard S. Sutton},
booktitle={Advances in Neural Information Processing Systems},
editor={A. Beygelzimer and Y. Dauphin and P. Liang and J. Wortman Vaughan},
year={2021},
url={https://openreview.net/forum?id=2w_2PwOYJarMu}
}