LENA: Communication-Efficient Distributed Learning with Self-Triggered Gradient Uploads
In distributed optimization, parameter updates from the gradient computing node devices have to be aggregated in every iteration on the orchestrating server. When these updates are sent over an arbitrary commodity network, bandwidth and latency can be limiting factors. We propose a communication fra…