ICML 2021spotlight49 citations
Robust Learning for Data Poisoning Attacks
Yunjuan Wang, Poorya Mianjy, Raman Arora
Abstract
We investigate the robustness of stochastic approximation approaches against data poisoning attacks. We focus on two-layer neural networks with ReLU activation and show that under a specific notion of separability in the RKHS induced by the infinite-width network, training (finite-width) networks with stochastic gradient descent is robust against data poisoning attacks. Interestingly, we find that in addition to a lower bound on the width of the network, which is standard in the literature, we also require a distribution-dependent upper bound on the width for robust generalization. We provide extensive empirical evaluations that support and validate our theoretical results.
BibTeX
@InProceedings{pmlr-v139-wang21r,
title = {Robust Learning for Data Poisoning Attacks},
author = {Wang, Yunjuan and Mianjy, Poorya and Arora, Raman},
booktitle = {Proceedings of the 38th International Conference on Machine Learning},
pages = {10859--10869},
year = {2021},
editor = {Meila, Marina and Zhang, Tong},
volume = {139},
series = {Proceedings of Machine Learning Research},
month = {18--24 Jul},
publisher = {PMLR},
pdf = {http://proceedings.mlr.press/v139/wang21r/wang21r.pdf},
url = {https://proceedings.mlr.press/v139/wang21r.html},
abstract = {We investigate the robustness of stochastic approximation approaches against data poisoning attacks. We focus on two-layer neural networks with ReLU activation and show that under a specific notion of separability in the RKHS induced by the infinite-width network, training (finite-width) networks with stochastic gradient descent is robust against data poisoning attacks. Interestingly, we find that in addition to a lower bound on the width of the network, which is standard in the literature, we also require a distribution-dependent upper bound on the width for robust generalization. We provide extensive empirical evaluations that support and validate our theoretical results.}
}