Defense Through Diverse Directions
Christopher Bender, Yang Li, Yifeng Shi, Michael K. Reiter, Junier Oliva
Abstract
In this work we develop a novel Bayesian neural network methodology to achieve strong adversarial robustness without the need for online adversarial training. Unlike previous efforts in this direction, we do not rely solely on the stochasticity of network weights by minimizing the divergence between the learned parameter distribution and a prior. Instead, we additionally require that the model maintain some expected uncertainty with respect to all input covariates. We demonstrate that by encouraging the network to distribute evenly across inputs, the network becomes less susceptible to localized, brittle features which imparts a natural robustness to targeted perturbations. We show empirical robustness on several benchmark datasets.
BibTeX
@InProceedings{pmlr-v119-bender20a,
title = {Defense Through Diverse Directions},
author = {Bender, Christopher and Li, Yang and Shi, Yifeng and Reiter, Michael K. and Oliva, Junier},
booktitle = {Proceedings of the 37th International Conference on Machine Learning},
pages = {756--766},
year = {2020},
editor = {III, Hal Daumé and Singh, Aarti},
volume = {119},
series = {Proceedings of Machine Learning Research},
month = {13--18 Jul},
publisher = {PMLR},
pdf = {http://proceedings.mlr.press/v119/bender20a/bender20a.pdf},
url = {https://proceedings.mlr.press/v119/bender20a.html},
abstract = {In this work we develop a novel Bayesian neural network methodology to achieve strong adversarial robustness without the need for online adversarial training. Unlike previous efforts in this direction, we do not rely solely on the stochasticity of network weights by minimizing the divergence between the learned parameter distribution and a prior. Instead, we additionally require that the model maintain some expected uncertainty with respect to all input covariates. We demonstrate that by encouraging the network to distribute evenly across inputs, the network becomes less susceptible to localized, brittle features which imparts a natural robustness to targeted perturbations. We show empirical robustness on several benchmark datasets.}
}