Collaborative Personalized Federated Learning via Exponential Moving Average Optimization

Abstract

Data heterogeneity poses a critical challenge in federated learning, driving the development of personalized client models. However, when each client’s local data is limited and nonindependent and identically distributed (non-IID), previous efforts fail to implement collaborative strategies based on the variability in data heterogeneity, thereby constraining the generalizability of the trained models. In this paper, we introduce a novel personalized Federated Learning method named FedCEMA, which tailors collaborative strategies to different scenarios of data heterogeneity. FedCEMA extracts prototype feature to align local objectives with the global optimum, and then calculates client-specific linear combinations of classifiers based on the similarity between clients.This allows clients to collaborate effectively by learning from those with similar data distributions, while maintaining personalized model benefits. Additionally, FedCEMA employs the Classifier Exponential Moving Average (CEMA) method, incorporating previously acquired knowledge to enhance the stability and efficiency of the model. Extensive experimental results on benchmark datasets demonstrate that our method improves both accuracy and stability compared to baselines.