Byzantine-Tolerant Methods for Distributed Variational Inequalities
Nazarii Tupitsa, Abdulla Jasem Almansoori, Yanlin Wu, Martin Takáč, Karthik Nandakumar, Samuel Horváth, Eduard Gorbunov
Abstract
Robustness to Byzantine attacks is a necessity for various distributed training scenarios. When the training reduces to the process of solving a minimization problem, Byzantine robustness is relatively well-understood. However, other problem formulations, such as min-max problems or, more generally, variational inequalities, arise in many modern machine learning and, in particular, distributed learning tasks. These problems significantly differ from the standard minimization ones and, therefore, require separate consideration. Nevertheless, only one work [Abidi et al., 2022] addresses this important question in the context of Byzantine robustness. Our work makes a further step in this direction by providing several (provably) Byzantine-robust methods for distributed variational inequality, thoroughly studying their theoretical convergence, removing the limitations of the previous work, and providing numerical comparisons supporting the theoretical findings.
BibTeX
@inproceedings{
tupitsa2023byzantinetolerant,
title={Byzantine-Tolerant Methods for Distributed Variational Inequalities},
author={Nazarii Tupitsa and Abdulla Jasem Almansoori and Yanlin Wu and Martin Tak{\'a}{\v{c}} and Karthik Nandakumar and Samuel Horv{\'a}th and Eduard Gorbunov},
booktitle={Thirty-seventh Conference on Neural Information Processing Systems},
year={2023},
url={https://openreview.net/forum?id=ER0bcYXvvo}
}