Albino Nogueiras Rodríguez

Automatic Adaption of the Step Size in Gradient Descent Training

Selecting an appropriate step size is critical in Gradient Descent algorithms used to train Neural Networks for Deep Learning tasks. A small value of the step size leads to slow convergence, while a large value can lead to instability. In this paper, we present ACME, a gradient descent algorithm tha…