NeurIPS 2018poster493 citations
Implicit Bias of Gradient Descent on Linear Convolutional Networks
Suriya Gunasekar, Jason Lee, Daniel Soudry, Nati Srebro
Abstract
We show that gradient descent on full-width linear convolutional networks of depth $L$ converges to a linear predictor related to the $\ell_{2/L}$ bridge penalty in the frequency domain. This is in contrast to linearly fully connected networks, where gradient descent converges to the hard margin linear SVM solution, regardless of depth.
BibTeX
@inproceedings{NEURIPS2018_0e98aeeb,
author = {Gunasekar, Suriya and Lee, Jason D and Soudry, Daniel and Srebro, Nati},
booktitle = {Advances in Neural Information Processing Systems},
editor = {S. Bengio and H. Wallach and H. Larochelle and K. Grauman and N. Cesa-Bianchi and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {Implicit Bias of Gradient Descent on Linear Convolutional Networks},
url = {https://proceedings.neurips.cc/paper_files/paper/2018/file/0e98aeeb54acf612b9eb4e48a269814c-Paper.pdf},
volume = {31},
year = {2018}
}