Beyond Euclidean Gossip: KL-Barycentric Consensus on Heterogeneous and Imbalanced Images
Fully decentralized deep learning removes global servers and ensures local data privacy. However, Euclidean consensus, averaging weights, gradients or momentum, may degrade under the situations with non-i.i.d. (non-independent and identically distributed) data and client size imbalance. We propose a