Anastasia Koloskova*

Decentralized Deep Learning with Arbitrary Communication Compression

Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks, as well as for efficient scaling to large compute clusters. As current approaches are limited by network bandwidth, we propose the use of communication compression in the d…