Quantitative convergence of trained neural networks to Gaussian processes

Andrea Agazzi; Eloy Mosig García; Dario Trevisan

NeurIPS 2025poster0 citations

Quantitative convergence of trained neural networks to Gaussian processes

Andrea Agazzi, Eloy Mosig García, Dario Trevisan

Abstract

In this paper, we study the quantitative convergence of shallow neural networks trained via gradient descent to their associated Gaussian processes in the infinite-width limit. While previous work has established qualitative convergence under broad settings, precise, finite-width estimates remain limited, particularly during training. We provide explicit upper bounds on the quadratic Wasserstein distance between the network output and its Gaussian approximation at any training time $t \ge 0$, demonstrating polynomial decay with network width. Our results quantify how architectural parameters, such as width and input dimension, influence convergence, and how training dynamics affect the approximation error

Neural networksNTKGaussian processWasserstein distanceGaussian approximationshallow neural networkswide limitinfinite-width neural network

BibTeX

@inproceedings{
agazzi2025quantitative,
title={Quantitative convergence of trained neural networks to Gaussian processes},
author={Andrea Agazzi and Eloy Mosig Garc{\'\i}a and Dario Trevisan},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
year={2025},
url={https://openreview.net/forum?id=sTyrh0LjoH}
}