ICLR 2017poster77 citations
A recurrent neural network without chaos
Thomas Laurent, James von Brecht
Abstract
We introduce an exceptionally simple gated recurrent neural network (RNN) that achieves performance comparable to well-known gated architectures, such as LSTMs and GRUs, on the word-level language modeling task. We prove that our model has simple, predicable and non-chaotic dynamics. This stands in stark contrast to more standard gated architectures, whose underlying dynamical systems exhibit chaotic behavior.
BibTeX
@inproceedings{
laurent2017a,
title={A recurrent neural network without chaos},
author={Thomas Laurent and James von Brecht},
booktitle={International Conference on Learning Representations},
year={2017},
url={https://openreview.net/forum?id=S1dIzvclg}
}