ICLR 2020poster173 citations
A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Case
Greg Ongie, Rebecca Willett, Daniel Soudry, Nathan Srebro
Abstract
We give a tight characterization of the (vectorized Euclidean) norm of weights required to realize a function $f:\mathbb{R}\rightarrow \mathbb{R}^d$ as a single hidden-layer ReLU network with an unbounded number of units (infinite width), extending the univariate characterization of Savarese et al. (2019) to the multivariate case.
inductive biasregularizationinfinite-width networksReLU networks
BibTeX
@inproceedings{
Ongie2020A,
title={A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Case},
author={Greg Ongie and Rebecca Willett and Daniel Soudry and Nathan Srebro},
booktitle={International Conference on Learning Representations},
year={2020},
url={https://openreview.net/forum?id=H1lNPxHKDH}
}