Jiawei Fei

SLAMB: Accelerated Large Batch Training with Sparse Communication

Distributed training of large deep neural networks requires frequent exchange of massive data between machines, thus communication efficiency is a major concern. Existing compressed communication methods are either not compatible with large batch optimization algorithms, or do not provide sufficient…