Reinforcement Learning through Asynchronous Advantage Actor-Critic on a GPU

Mohammad Babaeizadeh; Iuri Frosio; Stephen Tyree; Jason Clemons; Jan Kautz

ICLR 2017poster385 citations

Reinforcement Learning through Asynchronous Advantage Actor-Critic on a GPU

Mohammad Babaeizadeh, Iuri Frosio, Stephen Tyree, Jason Clemons, Jan Kautz

Abstract

We introduce a hybrid CPU/GPU version of the Asynchronous Advantage Actor-Critic (A3C) algorithm, currently the state-of-the-art method in reinforcement learning for various gaming tasks. We analyze its computational traits and concentrate on aspects critical to leveraging the GPU's computational power. We introduce a system of queues and a dynamic scheduling strategy, potentially helpful for other asynchronous algorithms as well. Our hybrid CPU/GPU version of A3C, based on TensorFlow, achieves a significant speed up compared to a CPU implementation; we make it publicly available to other researchers at https://github.com/NVlabs/GA3C.

Reinforcement Learning

BibTeX

@inproceedings{
babaeizadeh2017reinforcement,
title={Reinforcement Learning through Asynchronous Advantage Actor-Critic on a {GPU}},
author={Mohammad Babaeizadeh and Iuri Frosio and Stephen Tyree and Jason Clemons and Jan Kautz},
booktitle={International Conference on Learning Representations},
year={2017},
url={https://openreview.net/forum?id=r1VGvBcxl}
}