The Importance of Sampling inMeta-Reinforcement Learning
Bradly Stadie, Ge Yang, Rein Houthooft, Peter Chen, Yan Duan, Yuhuai Wu, Pieter Abbeel, Ilya Sutskever
Abstract
We interpret meta-reinforcement learning as the problem of learning how to quickly find a good sampling distribution in a new environment. This interpretation leads to the development of two new meta-reinforcement learning algorithms: E-MAML and E-$\text{RL}^2$. Results are presented on a new environment we call `Krazy World': a difficult high-dimensional gridworld which is designed to highlight the importance of correctly differentiating through sampling distributions in meta-reinforcement learning. Further results are presented on a set of maze environments. We show E-MAML and E-$\text{RL}^2$ deliver better performance than baseline algorithms on both tasks.
BibTeX
@inproceedings{NEURIPS2018_d0f5722f,
author = {Stadie, Bradly and Yang, Ge and Houthooft, Rein and Chen, Peter and Duan, Yan and Wu, Yuhuai and Abbeel, Pieter and Sutskever, Ilya},
booktitle = {Advances in Neural Information Processing Systems},
editor = {S. Bengio and H. Wallach and H. Larochelle and K. Grauman and N. Cesa-Bianchi and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {The Importance of Sampling inMeta-Reinforcement Learning},
url = {https://proceedings.neurips.cc/paper_files/paper/2018/file/d0f5722f11a0cc839fa2ca6ea49d8585-Paper.pdf},
volume = {31},
year = {2018}
}