Deep Residual Reinforcement Learning (Extended Abstract)

Shangtong Zhang; Wendelin Boehmer; Shimon Whiteson

IJCAI 2021poster0 citations

Deep Residual Reinforcement Learning (Extended Abstract)

Shangtong Zhang, Wendelin Boehmer, Shimon Whiteson

Abstract

We revisit residual algorithms in both model-free and model-based reinforcement learning settings. We propose the bidirectional target network technique to stabilize residual algorithms, yielding a residual version of DDPG that significantly outperforms vanilla DDPG in commonly used benchmarks. Moreover, we find the residual algorithm an effective approach to the distribution mismatch problem in model-based planning. Compared with the existing TD(k) method, our residual-based method makes weaker assumptions about the model and yields a greater performance boost.

Machine Learning: Reinforcement Learning

BibTeX

@inproceedings{ijcai2021p668,
  title     = {Deep Residual Reinforcement Learning (Extended Abstract)},
  author    = {Zhang, Shangtong and Boehmer, Wendelin and Whiteson, Shimon},
  booktitle = {Proceedings of the Thirtieth International Joint Conference on
               Artificial Intelligence, {IJCAI-21}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Zhi-Hua Zhou},
  pages     = {4869--4873},
  year      = {2021},
  month     = {8},
  note      = {Sister Conferences Best Papers},
  doi       = {10.24963/ijcai.2021/668},
  url       = {https://doi.org/10.24963/ijcai.2021/668},
}