Annie N. Wang

Plateau in Monotonic Linear Interpolation --- A "Biased" View of Loss Landscape for Deep Networks

Monotonic linear interpolation (MLI) --- on the line connecting a random initialization with the minimizer it converges to, the loss and accuracy are monotonic --- is a phenomenon that is commonly observed in the training of neural networks. Such a phenomenon may seem to suggest that optimization o…