Lior Kreimer

Revisiting the Volume Hypothesis

Modern deep neural networks often contain far more parameters than needed to fit their training data, yet they achieve impressive generalization. A common explanation of this success is the implicit bias of stochastic gradient descent (SGD). An alternative volume hypothesis posits that, within low t…