Sparse DNNs with Improved Adversarial Robustness
Yiwen Guo, Chao Zhang, Changshui Zhang, Yurong Chen
Abstract
Deep neural networks (DNNs) are computationally/memory-intensive and vulnerable to adversarial attacks, making them prohibitive in some real-world applications. By converting dense models into sparse ones, pruning appears to be a promising solution to reducing the computation/memory cost. This paper studies classification models, especially DNN-based ones, to demonstrate that there exists intrinsic relationships between their sparsity and adversarial robustness. Our analyses reveal, both theoretically and empirically, that nonlinear DNN-based classifiers behave differently under $l_2$ attacks from some linear ones. We further demonstrate that an appropriately higher model sparsity implies better robustness of nonlinear DNNs, whereas over-sparsified models can be more difficult to resist adversarial examples.
BibTeX
@inproceedings{NEURIPS2018_4c5bde74,
author = {Guo, Yiwen and Zhang, Chao and Zhang, Changshui and Chen, Yurong},
booktitle = {Advances in Neural Information Processing Systems},
editor = {S. Bengio and H. Wallach and H. Larochelle and K. Grauman and N. Cesa-Bianchi and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {Sparse DNNs with Improved Adversarial Robustness},
url = {https://proceedings.neurips.cc/paper_files/paper/2018/file/4c5bde74a8f110656874902f07378009-Paper.pdf},
volume = {31},
year = {2018}
}