Invariance-inducing regularization using worst-case transformations suffices to boost accuracy and spatial robustness
Fanny Yang, Zuowen Wang, Christina Heinze-Deml
Abstract
This work provides theoretical and empirical evidence that invariance-inducing regularizers can increase predictive accuracy for worst-case spatial transformations (spatial robustness). Evaluated on these adversarially transformed examples, standard and adversarial training with such regularizers achieves a relative error reduction of 20% for CIFAR-10 with the same computational budget. This even surpasses handcrafted spatial-equivariant networks. Furthermore, we observe for SVHN, known to have inherent variance in orientation, that robust training also improves standard accuracy on the test set. We prove that this no-trade-off phenomenon holds for adversarial examples from transformation groups.
BibTeX
@inproceedings{NEURIPS2019_1d01bd2e,
author = {Yang, Fanny and Wang, Zuowen and Heinze-Deml, Christina},
booktitle = {Advances in Neural Information Processing Systems},
editor = {H. Wallach and H. Larochelle and A. Beygelzimer and F. d\textquotesingle Alch\'{e}-Buc and E. Fox and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {Invariance-inducing regularization using worst-case transformations suffices to boost accuracy and spatial robustness},
url = {https://proceedings.neurips.cc/paper_files/paper/2019/file/1d01bd2e16f57892f0954902899f0692-Paper.pdf},
volume = {32},
year = {2019}
}