Wang Kong

Fast Convergence of Natural Gradient Descent for Over-parameterized Physics-Informed Neural Networks

In the context of over-parameterization, there is a line of work demonstrating that randomly initialized (stochastic) gradient descent (GD) converges to a globally optimal solution at a linear convergence rate for the quadratic loss function. However, the convergence rate of GD for training two-laye…