2015
Reducing communication overhead in distributed learning by an order of magnitude (almost)
ICASSP 2015accepted
Large-scale distributed learning plays an ever-more increasing role in modern computing. However, whether using a compute cluster with thousands of nodes, or a single multi-GPU machine, the most significant bottleneck is that of communication. In this work, we explore the effects of applying quantiz…