Can Fairness and Robustness Be Simultaneously Achieved Under Byzantine Attacks?

Abstract

Fairness among different workers and robustness to Byzantine attacks are two critical issues in distributed learning. In this paper, we attempt to answer the following question: Can we simultaneously achieve fairness and robustness under Byzantine attacks? Here we provide a negative answer: It is very difficult to kill two birds with one stone. First, we observe that most of the existing robust distributed learning algorithms rely on robust aggregators to aggregate messages from the workers, and such robust aggregators share a common majority-dominance property. Second, we prove that a class of fair distributed learning algorithms, replacing the mean aggregator by those robust aggregators having the majority-dominance property to enhance robustness to Byzantine attacks, lead to unfair solutions even for a simple distributed linear regression problem. Third, we conduct numerical experiments on distributed linear regression and nonlinear classification, showing these algorithms to be either short of fairness or lack of robustness.