Lifang Chen

Optimal Distributed Training With Co-Adaptive Data Parallelism in Heterogeneous Environments

The computational power required for training deep learning models has been skyrocketing in the past decade as they scale with big data, and has become a very expensive and scarce resource. Therefore, distributed training, which can leverage distributed available computational power, is vital for ef