NeurIPS 2021spotlight33 citations
A single gradient step finds adversarial examples on random two-layers neural networks
Sebastien Bubeck, Yeshwanth Cherapanamjeri, Gauthier Gidel, Remi Tachet des Combes
Abstract
Daniely and Schacham recently showed that gradient descent finds adversarial examples on random undercomplete two-layers ReLU neural networks. The term “undercomplete” refers to the fact that their proof only holds when the number of neurons is a vanishing fraction of the ambient dimension. We extend their result to the overcomplete case, where the number of neurons is larger than the dimension (yet also subexponential in the dimension). In fact we prove that a single step of gradient descent suffices. We also show this result for any subexponential width random neural network with smooth activation function.
adversarial examplesrandom neural networksdeep-learning theory
BibTeX
@inproceedings{
bubeck2021a,
title={A single gradient step finds adversarial examples on random two-layers neural networks},
author={Sebastien Bubeck and Yeshwanth Cherapanamjeri and Gauthier Gidel and Remi Tachet des Combes},
booktitle={Advances in Neural Information Processing Systems},
editor={A. Beygelzimer and Y. Dauphin and P. Liang and J. Wortman Vaughan},
year={2021},
url={https://openreview.net/forum?id=NE0YlkgRo9x}
}