Hamid Reza Feyzmahdavian

Delay-Adaptive Step-sizes for Asynchronous Learning

In scalable machine learning systems, model training is often parallelized over multiple nodes that run without tight synchronization. Most analysis results for the related asynchronous algorithms use an upper bound on the information delays in the system to determine learning rates. Not only are su…