Rejection of Workers with Heterogeneous (Mismatched) Data in Federated Learning *

Abstract

Federated learning (FL) has attracted great attention lately due to many advantages it provides. The community has noted that FL performance can be greatly degraded by heterogeneous data or any data that is unsuitable for the learning problem under consideration. Here we focus on the gradient descent and heavy ball algorithms in a worker-server architecture and describe a simple but effective approach to reject data from workers with unsuitable data that would significantly harm the training performance. The approach uses a very short pre-training phase where only a few workers who have known suitable data are involved. Using the theory of order statistics, this pre-training allows accurate estimation of the range of gradient directions that should be present during the initial iterations of training, allowing workers with unsuitable data to be recognized. The approach is tested on real data sets and shows promising results for all cases tested.