NeurIPS 2019poster48 citations
Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias
Stéphane d'Ascoli, Levent Sagun, Giulio Biroli, Joan Bruna
Abstract
Despite the phenomenal success of deep neural networks in a broad range of learning tasks, there is a lack of theory to understand the way they work. In particular, Convolutional Neural Networks (CNNs) are known to perform much better than Fully-Connected Networks (FCNs) on spatially structured data: the architectural structure of CNNs benefits from prior knowledge on the features of the data, for instance their translation invariance. The aim of this work is to understand this fact through the lens of dynamics in the loss landscape.
BibTeX
@inproceedings{NEURIPS2019_124c3e4a,
author = {d\textquotesingle Ascoli, St\'{e}phane and Sagun, Levent and Biroli, Giulio and Bruna, Joan},
booktitle = {Advances in Neural Information Processing Systems},
editor = {H. Wallach and H. Larochelle and A. Beygelzimer and F. d\textquotesingle Alch\'{e}-Buc and E. Fox and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias},
url = {https://proceedings.neurips.cc/paper_files/paper/2019/file/124c3e4ada4a529aa0fedece80bb42ab-Paper.pdf},
volume = {32},
year = {2019}
}