Robust Reinforcement Learning via Adversarial training with Langevin Dynamics
Parameswaran Kamalaruban, Yu-Ting Huang, Ya-Ping Hsieh, Paul Rolland, Cheng Shi, Volkan Cevher
Abstract
We introduce a \emph{sampling} perspective to tackle the challenging task of training robust Reinforcement Learning (RL) agents. Leveraging the powerful Stochastic Gradient Langevin Dynamics, we present a novel, scalable two-player RL algorithm, which is a sampling variant of the two-player policy gradient method. Our algorithm consistently outperforms existing baselines, in terms of generalization across different training and testing conditions, on several MuJoCo environments. Our experiments also show that, even for objective functions that entirely ignore potential environmental shifts, our sampling approach remains highly robust in comparison to standard RL algorithms.
BibTeX
@inproceedings{NEURIPS2020_5cb0e249,
author = {Kamalaruban, Parameswaran and Huang, Yu-Ting and Hsieh, Ya-Ping and Rolland, Paul and Shi, Cheng and Cevher, Volkan},
booktitle = {Advances in Neural Information Processing Systems},
editor = {H. Larochelle and M. Ranzato and R. Hadsell and M.F. Balcan and H. Lin},
pages = {8127--8138},
publisher = {Curran Associates, Inc.},
title = {Robust Reinforcement Learning via Adversarial training with Langevin Dynamics},
url = {https://proceedings.neurips.cc/paper_files/paper/2020/file/5cb0e249689cd6d8369c4885435a56c2-Paper.pdf},
volume = {33},
year = {2020}
}