Heudson Mirandola

Get rid of your constraints and reparametrize: A study in NNLS and implicit bias

Over the past years, there has been significant interest in understanding the implicit bias of gradient descent optimization and its connection to the generalization properties of overparametrized neural networks. Several works observed that when training linear diagonal networks on the square loss…