Refined Analysis of Constant Step Size Federated Averaging and Federated Richardson-Romberg Extrapolation
Paul Mangold, Alain Oliviero Durmus, Aymeric Dieuleveut, Sergey Samsonov, Eric Moulines
Abstract
In this paper, we present a novel analysis of $\texttt{FedAvg}$ with constant step size, relying on the Markov property of the underlying process. We demonstrate that the global iterates of the algorithm converge to a stationary distribution and analyze its resulting bias and variance relative to the problem's solution. We provide a first-order bias expansion in both homogeneous and heterogeneous settings. Interestingly, this bias decomposes into two distinct components: one that depends solely on stochastic gradient noise and another on client heterogeneity. Finally, we introduce a new algorithm based on the Richardson-Romberg extrapolation technique to mitigate this bias.
BibTeX
@inproceedings{
mangold2025refined,
title={Refined Analysis of Constant Step Size Federated Averaging},
author={Paul Mangold and Alain Oliviero Durmus and Aymeric Dieuleveut and Sergey Samsonov and Eric Moulines},
booktitle={The 28th International Conference on Artificial Intelligence and Statistics},
year={2025},
url={https://openreview.net/forum?id=YxgThbmXgN}
}