Julius Girardin

Scaling Laws and Spectra of Shallow Neural Networks in the Feature Learning Regime

Neural scaling laws underlie many of the recent advances in deep learning, yet their theoretical understanding remains largely confined to linear models. In this work, we present a systematic analysis of scaling laws for quadratic and diagonal neural networks in the feature learning regime. Leveragi…