Learning Hierarchical Information Flow with Recurrent Neural Modules
Danijar Hafner, Alexander Irpan, James Davidson, Nicolas Heess
Abstract
We propose ThalNet, a deep learning model inspired by neocortical communication via the thalamus. Our model consists of recurrent neural modules that send features through a routing center, endowing the modules with the flexibility to share features over multiple time steps. We show that our model learns to route information hierarchically, processing input data by a chain of modules. We observe common architectures, such as feed forward neural networks and skip connections, emerging as special cases of our architecture, while novel connectivity patterns are learned for the text8 compression task. Our model outperforms standard recurrent neural networks on several sequential benchmarks.
BibTeX
@inproceedings{NIPS2017_6c349155,
author = {Hafner, Danijar and Irpan, Alexander and Davidson, James and Heess, Nicolas},
booktitle = {Advances in Neural Information Processing Systems},
editor = {I. Guyon and U. Von Luxburg and S. Bengio and H. Wallach and R. Fergus and S. Vishwanathan and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {Learning Hierarchical Information Flow with Recurrent Neural Modules},
url = {https://proceedings.neurips.cc/paper_files/paper/2017/file/6c349155b122aa8ad5c877007e05f24f-Paper.pdf},
volume = {30},
year = {2017}
}