Imagination-Augmented Agents for Deep Reinforcement Learning
Sébastien Racanière, Theophane Weber, David Reichert, Lars Buesing, Arthur Guez, Danilo Jimenez Rezende, Adrià Puigdomènech Badia, Oriol Vinyals
Abstract
We introduce Imagination-Augmented Agents (I2As), a novel architecture for deep reinforcement learning combining model-free and model-based aspects. In contrast to most existing model-based reinforcement learning and planning methods, which prescribe how a model should be used to arrive at a policy, I2As learn to interpret predictions from a trained environment model to construct implicit plans in arbitrary ways, by using the predictions as additional context in deep policy networks. I2As show improved data efficiency, performance, and robustness to model misspecification compared to several strong baselines.
BibTeX
@inproceedings{NIPS2017_9e82757e,
author = {Racani\`{e}re, S\'{e}bastien and Weber, Theophane and Reichert, David and Buesing, Lars and Guez, Arthur and Jimenez Rezende, Danilo and Puigdom\`{e}nech Badia, Adri\`{a} and Vinyals, Oriol and Heess, Nicolas and Li, Yujia and Pascanu, Razvan and Battaglia, Peter and Hassabis, Demis and Silver, David and Wierstra, Daan},
booktitle = {Advances in Neural Information Processing Systems},
editor = {I. Guyon and U. Von Luxburg and S. Bengio and H. Wallach and R. Fergus and S. Vishwanathan and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {Imagination-Augmented Agents for Deep Reinforcement Learning},
url = {https://proceedings.neurips.cc/paper_files/paper/2017/file/9e82757e9a1c12cb710ad680db11f6f1-Paper.pdf},
volume = {30},
year = {2017}
}