2022
Implicit Gradient Alignment in Distributed and Federated Learning
AAAI 2022technical
A major obstacle to achieving global convergence in distributed and federated learning is the misalignment of gradients across clients or mini-batches due to heterogeneity and stochasticity of the distributed data. In this work, we show that data heterogeneity can in fact be exploited to improve gen…