NeurIPS 2018oral14 citations
On Neuronal Capacity
Abstract
We define the capacity of a learning machine to be the logarithm of the number (or volume) of the functions it can implement. We review known results, and derive new results, estimating the capacity of several neuronal models: linear and polynomial threshold gates, linear and polynomial threshold gates with constrained weights (binary weights, positive weights), and ReLU neurons. We also derive capacity estimates and bounds for fully recurrent networks and layered feedforward networks.
BibTeX
@inproceedings{NEURIPS2018_a292f1c5,
author = {Baldi, Pierre and Vershynin, Roman},
booktitle = {Advances in Neural Information Processing Systems},
editor = {S. Bengio and H. Wallach and H. Larochelle and K. Grauman and N. Cesa-Bianchi and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {On Neuronal Capacity},
url = {https://proceedings.neurips.cc/paper_files/paper/2018/file/a292f1c5874b2be8395ffd75f313937f-Paper.pdf},
volume = {31},
year = {2018}
}