2021
Continuous vs. Discrete Optimization of Deep Neural Networks
NeurIPS 2021spotlight
Existing analyses of optimization in deep learning are either continuous, focusing on (variants of) gradient flow, or discrete, directly treating (variants of) gradient descent. Gradient flow is amenable to theoretical analysis, but is stylized and disregards computational efficiency. The extent t…