Frank-Wolfe Method with Proximal Regularization for Constrained Federated Learning with Non-iid Data

Abstract

Federated constrained learning allows us to learn a global model with some specific structure to enhance performance. Most existing federated learning techniques assume data is homogeneously or independently and identically distributed (iid) across clients. However, this iid assumption rarely holds in practice, and the averaging operation on the server’s side leads to degradation in performance at the client level. In this paper, we propose FedFW-Prox, a novel and computationally efficient federated Frank-Wolfe algorithm that learns personalized models for each client, which guarantees better performance even with heterogeneous data distributions across clients. Instead of modifying the objective function, we introduce a proximal regularization term at the update step at each client, simplifying the update process. This regularization term ensures that local models remain close to the global model, promoting convergence to the global optimal point rather than the local optima. The proposed method guarantees a sublinear convergence of $\mathcal{O}(1/\sqrt k )$ for smooth convex objective functions, where k denotes the iteration number. We empirically evaluate the effectiveness of the proposed approach on various machine-learning tasks.