AAAI 2022technical40 citations
Policy Optimization with Stochastic Mirror Descent
Long Yang, Yu Zhang, Gang Zheng, Qian Zheng, Pengfei Li, Jianhang Huang, Gang Pan
Abstract
Improving sample efficiency has been a longstanding goal in reinforcement learning. This paper proposes VRMPO algorithm: a sample efficient policy gradient method with stochastic mirror descent. In VRMPO, a novel variance-reduced policy gradient estimator is presented to improve sample efficiency. We prove that the proposed VRMPO needs only O(ε−3) sample trajectories to achieve an ε-approximate first-order stationary point, which matches the best sample complexity for policy optimization. Extensive empirical results demonstrate that VRMP outperforms the state-of-the-art policy gradient methods in various settings.
BibTeX
@inproceedings{aaai2022_policyoptimizati,
title = {Policy Optimization with Stochastic Mirror Descent},
author = {Long Yang and Yu Zhang and Gang Zheng and Qian Zheng and Pengfei Li and Jianhang Huang and Gang Pan},
booktitle = {AAAI 2022},
year = {2022}
}