Rodrigo Veiga — accepted AI-conference papers

2024

Stochastic Gradient Flow Dynamics of Test Risk and its Exact Solution for Weak Features

Rodrigo Veiga, Anastasia Remizova, Nicolas Macris

ICML 2024poster

We investigate the test risk of a continuous time stochastic gradient flow dynamics in learning theory. Using a path integral formulation we provide, in the regime of small learning rate, a general formula for computing the difference between test risk curves of pure gradient and stochastic gradient…

Cited by 3 Source ↗PDF ↗code ↗Scholar ↗

2022

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

Rodrigo Veiga, Ludovic STEPHAN, Bruno Loureiro, Florent Krzakala, Lenka Zdeborova

NeurIPS 2022accept

Despite the non-convex optimization landscape, over-parametrized shallow networks are able to achieve global convergence under gradient descent. The picture can be radically different for narrow networks, which tend to get stuck in badly-generalizing local minima. Here we investigate the cross-over…

Cited by 49 Source ↗PDF ↗code ↗Scholar ↗