Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learning
Anis Elgabli, Jihong Park, Amrit S. Bedi, Mehdi Bennis, Vaneet Aggarwal
Abstract
In this paper, we propose a communication-efficient decen-tralized machine learning (ML) algorithm, coined quantized group ADMM (Q-GADMM). Every worker in Q-GADMM communicates only with two neighbors, and updates its model via the group alternating direct method of multiplier (GADMM), thereby ensuring fast convergence while reducing the number of communication rounds. Furthermore, each worker quantizes its model updates before transmissions, thereby decreasing the communication payload sizes. We prove that Q-GADMM converges to the optimal solution for convex loss functions, and numerically show that Q-GADMM yields 7x less communication cost while achieving almost the same accuracy and convergence speed compared to GADMM without quantization.
BibTeX
@inproceedings{icassp2020_qgadmmquantizedg,
title = {Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learning},
author = {Anis Elgabli and Jihong Park and Amrit S. Bedi and Mehdi Bennis and Vaneet Aggarwal},
booktitle = {ICASSP 2020},
year = {2020}
}