AdaGK-SGD: Adaptive Global Knowledge Guided Distributed Stochastic Gradient Descent
Distributed machine learning (DML) is promising for training large models on large datasets. In DML, multiple workers collaborate on the training of neural networks, significantly reducing the time required for neural network training. The efficiency of DML is heavily influenced by communication,…