Võ Thục Khánh Huyền

Improving Neural Ordinary Differential Equations with Nesterov's Accelerated Gradient Method

We propose the Nesterov neural ordinary differential equations (NesterovNODEs), whose layers solve the second-order ordinary differential equations (ODEs) limit of Nesterov's accelerated gradient (NAG) method, and a generalization called GNesterovNODEs. Taking the advantage of the convergence rate $…