ICLR 2019poster234 citations
The Singular Values of Convolutional Layers
Hanie Sedghi, Vineet Gupta, Philip M. Long
Abstract
We characterize the singular values of the linear transformation associated with a standard 2D multi-channel convolutional layer, enabling their efficient computation. This characterization also leads to an algorithm for projecting a convolutional layer onto an operator-norm ball. We show that this is an effective regularizer; for example, it improves the test error of a deep residual network using batch normalization on CIFAR-10 from 6.2% to 5.3%.
singular valuesoperator normconvolutional layersregularization
BibTeX
@inproceedings{
sedghi2018the,
title={The Singular Values of Convolutional Layers},
author={Hanie Sedghi and Vineet Gupta and Philip M. Long},
booktitle={International Conference on Learning Representations},
year={2019},
url={https://openreview.net/forum?id=rJevYoA9Fm},
}