Robin M Schmidt

Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers

Choosing the optimizer is considered to be among the most crucial design decisions in deep learning, and it is not an easy one. The growing literature now lists hundreds of optimization methods. In the absence of clear theoretical guidance and conclusive empirical evidence, the decision is often mad…