Yudong Wei

On the Benefits of Weight Normalization for Overparameterized Matrix Sensing

While normalization techniques are widely used in deep learning, their theoretical understanding remains relatively limited. In this work, we establish the benefits of (generalized) weight normalization (WN) applied to the overparameterized matrix sensing problem. We prove that WN with Riemannian op…