When Green Learning Meets Federated Learning: Toward Distributed Learning with Low Complexity and Model Heterogeneity

Abstract

In this paper, we study the problem of aggregating heterogeneous models with low-complexity consideration in federated learning (FL). A green heterogeneous federated learning (GHFL) system, the marriage of the green learning (GL) methodology and FL, is proposed to simultaneously account for model heterogeneity and computation complexity. The unique properties of scalability and modularized design in GL are leveraged to facilitate low-complexity and personalized learning models in local clients which still can be aggregated at the server. Particularly, inspired by the subspace approximation technique developed in GL, we devise an aggregation rule by leveraging the distributed multi-stage principal component analysis (PCA) in this work. Prominent anchor vectors of local data spaces from heterogeneous local clients can be properly and reliably aggregated at the server. Numerical experiments validate the effectiveness of the proposed GHFL.