Jianxing Wang

Gradient Flow Dynamics and Implicit Bias of Diagonal Linear Networks under Infinitesimal Initialization

We study the gradient flow dynamics of diagonal linear networks for regression tasks under infinitesimal initialization. Extending the saddle-to-saddle dynamics described in Theorem 1 from Pesme & Flammarion (2023), we generalize the analysis to both deep diagonal linear networks and a broader class…