ICML 2016poster371 citations
Graying the black box: Understanding DQNs
Tom Zahavy, Nir Ben-Zrihem, Shie Mannor
Abstract
In recent years there is a growing interest in using deep representations for reinforcement learning. In this paper, we present a methodology and tools to analyze Deep Q-networks (DQNs) in a non-blind matter. Using our tools we reveal that the features learned by DQNs aggregate the state space in a hierarchical fashion, explaining its success. Moreover we are able to understand and describe the policies learned by DQNs for three different Atari2600 games and suggest ways to interpret, debug and optimize of deep neural networks in Reinforcement Learning.
BibTeX
@InProceedings{pmlr-v48-zahavy16,
title = {Graying the black box: Understanding DQNs},
author = {Zahavy, Tom and Ben-Zrihem, Nir and Mannor, Shie},
booktitle = {Proceedings of The 33rd International Conference on Machine Learning},
pages = {1899--1908},
year = {2016},
editor = {Balcan, Maria Florina and Weinberger, Kilian Q.},
volume = {48},
series = {Proceedings of Machine Learning Research},
address = {New York, New York, USA},
month = {20--22 Jun},
publisher = {PMLR},
pdf = {http://proceedings.mlr.press/v48/zahavy16.pdf},
url = {https://proceedings.mlr.press/v48/zahavy16.html},
abstract = {In recent years there is a growing interest in using deep representations for reinforcement learning. In this paper, we present a methodology and tools to analyze Deep Q-networks (DQNs) in a non-blind matter. Using our tools we reveal that the features learned by DQNs aggregate the state space in a hierarchical fashion, explaining its success. Moreover we are able to understand and describe the policies learned by DQNs for three different Atari2600 games and suggest ways to interpret, debug and optimize of deep neural networks in Reinforcement Learning.}
}