NeurIPS 2019poster69 citations
Can SGD Learn Recurrent Neural Networks with Provable Generalization?
Abstract
Recurrent Neural Networks (RNNs) are among the most popular models in sequential data analysis. Yet, in the foundational PAC learning language, what concept class can it learn? Moreover, how can the same recurrent unit simultaneously learn functions from different input tokens to different output tokens, without affecting each other? Existing generalization bounds for RNN scale exponentially with the input length, significantly limiting their practical implications.
BibTeX
@inproceedings{NEURIPS2019_67fe0f66,
author = {Allen-Zhu, Zeyuan and Li, Yuanzhi},
booktitle = {Advances in Neural Information Processing Systems},
editor = {H. Wallach and H. Larochelle and A. Beygelzimer and F. d\textquotesingle Alch\'{e}-Buc and E. Fox and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {Can SGD Learn Recurrent Neural Networks with Provable Generalization?},
url = {https://proceedings.neurips.cc/paper_files/paper/2019/file/67fe0f66449e31fdafdc3505c37d6acb-Paper.pdf},
volume = {32},
year = {2019}
}