HFedPFS: Heterogeneous Federated Learning with Personalized Data Feature Sharing

Abstract

Federated learning (FL) is a distributed machine learning technique enabling multiple clients to jointly train a global model while preserving the privacy of their non-IID (non-independent and identically) data. However, traditional FL approaches require clients to use the same model structure as the global model, which is not suitable for scenarios where clients need to train heterogeneous local models with different architectures, known as Heterogeneous Federated Learning (HFL). Current HFL approaches often exchange all features of local data through the interaction of the client model and the server model. The local data from different clients may present similar generic features, and sharing them over clients may hinder the effective learning of personalized features which truly leads to non-IID distributions. To facilitate the effective information exchange between the server and client while maintaining efficient communication and computation, we propose a novel Heterogeneous Federated learning method (HFedPFS) based on Personalized data Feature Sharing. We designed two significant patterns for this algorithm: (1) A Feature Perception Network (FPN) separates generic and personalized features at each client. (2) A global homogeneous adapter processes the personalized features, enabling effective bidirectional feature exchange. HFedPFS outperforms six state-of-the-art HFL methods on two datasets, improving accuracy by up to 3.54 and 4.17 percentage points in homogeneous and heterogeneous scenarios, respectively while reducing training time by 29.7%.