Curriculum-guided Hindsight Experience Replay
Meng Fang, Tianyi Zhou, Yali Du, Lei Han, Zhengyou Zhang
Abstract
In off-policy deep reinforcement learning, it is usually hard to collect sufficient successful experiences with sparse rewards to learn from. Hindsight experience replay (HER) enables an agent to learn from failures by treating the achieved state of a failed experience as a pseudo goal. However, not all the failed experiences are equally useful to different learning stages, so it is not efficient to replay all of them or uniform samples of them. In this paper, we propose to 1) adaptively select the failed experiences for replay according to the proximity to the true goals and the curiosity of exploration over diverse pseudo goals, and 2) gradually change the proportion of the goal-proximity and the diversity-based curiosity in the selection criteria: we adopt a human-like learning strategy that enforces more curiosity in earlier stages and changes to larger goal-proximity later. This
BibTeX
@inproceedings{NEURIPS2019_83715fd4,
author = {Fang, Meng and Zhou, Tianyi and Du, Yali and Han, Lei and Zhang, Zhengyou},
booktitle = {Advances in Neural Information Processing Systems},
editor = {H. Wallach and H. Larochelle and A. Beygelzimer and F. d\textquotesingle Alch\'{e}-Buc and E. Fox and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {Curriculum-guided Hindsight Experience Replay},
url = {https://proceedings.neurips.cc/paper_files/paper/2019/file/83715fd4755b33f9c3958e1a9ee221e1-Paper.pdf},
volume = {32},
year = {2019}
}