FedCAda: Adaptive Client-Side Optimization for Accelerated and Stable Federated Learning

Abstract

Federated learning (FL) enables collaborative model training across distributed clients while preserving data privacy. However, achieving both acceleration and stability, particularly on the client side, remains a challenge. In this paper, we introduce FedCAda, an adaptive algorithm that leverages an Adam-like approach to adjust first and second moment estimates on the client side while aggregating adaptive parameters on the server side. This design aims to accelerate convergence without compromising stability and performance. We also explore several algorithms with different adjustment functions and find that stronger constraints on adaptive parameters are necessary in the early stages of FL, when information from other clients is limited. Experiments on public datasets demonstrate that FedCAda surpasses state-of-the-art methods in adaptability, convergence, and stability, advancing adaptive algorithms for FL.