FedSe: Group-Based Sequential Training Strategies for Mitigating Label Skew in Federated Learning

Abstract

Federated Learning (FL) has emerged as a promising approach for distributed machine learning, enabling clients to collaboratively train models without sharing their data. However, existing FL methods continue to face challenges when dealing with non-IID data, particularly under conditions of extreme label skew. This divergence among client models can lead to a significant degradation in the accuracy of the global model. To address this critical issue, we introduce the FedSe approach, which incorporates two novel components: homogeneous grouping and sequential training. First, clients are grouped based on the distribution of data labels to ensure that each group contains a balanced representation of all labels. Second, task models are trained sequentially within these groups while training occurs in parallel across groups. The final step involves aggregating the models through averaging. Extensive experimental results on several datasets with label skew demonstrate the effectiveness of FedSe, and its results surpass current state-of-the-art methods.