Unfolding recurrence by Green’s functions for optimized reservoir computing
Sandra Nestler, Christian Keup, David Dahmen, Matthieu Gilson, Holger Rauhut, Moritz Helias
Abstract
Cortical networks are strongly recurrent, and neurons have intrinsic temporal dynamics. This sets them apart from deep feed-forward networks. Despite the tremendous progress in the application of deep feed-forward networks and their theoretical understanding, it remains unclear how the interplay of recurrence and non-linearities in recurrent cortical networks contributes to their function. The purpose of this work is to present a solvable recurrent network model that links to feed forward networks. By perturbative methods we transform the time-continuous, recurrent dynamics into an effective feed-forward structure of linear and non-linear temporal kernels. The resulting analytical expressions allow us to build optimal time-series classifiers from random reservoir networks. Firstly, this allows us to optimize not only the readout vectors, but also the input projection, demonstrating a strong potential performance gain. Secondly, the analysis exposes how the second order stimulus statistics is a crucial element that interacts with the non-linearity of the dynamics and boosts performance.
BibTeX
@inproceedings{NEURIPS2020_c94a589b,
author = {Nestler, Sandra and Keup, Christian and Dahmen, David and Gilson, Matthieu and Rauhut, Holger and Helias, Moritz},
booktitle = {Advances in Neural Information Processing Systems},
editor = {H. Larochelle and M. Ranzato and R. Hadsell and M.F. Balcan and H. Lin},
pages = {17380--17390},
publisher = {Curran Associates, Inc.},
title = {Unfolding recurrence by Green’s functions for optimized reservoir computing},
url = {https://proceedings.neurips.cc/paper_files/paper/2020/file/c94a589bdd47870b1d74b258d1ce3b33-Paper.pdf},
volume = {33},
year = {2020}
}