Chandra Shekar Lakshminarayanan

Deep Networks Learn Features From Local Discontinuities in the Label Function

Deep neural networks outperform kernel machines on several datasets due to feature learning that happens during gradient descent training. In this paper, we analyze the mechanism through which feature learning happens and use a notion of features that corresponds to discontinuities in the true label…