Karl Bäckström

ASAP.SGD: Instance-based Adaptiveness to Staleness in Asynchronous SGD

Concurrent algorithmic implementations of Stochastic Gradient Descent (SGD) give rise to critical questions for compute-intensive Machine Learning (ML). Asynchrony implies speedup in some contexts, and challenges in others, as stale updates may lead to slower, or non-converging executions. While pre…