Roman Worschech

Analyzing Neural Scaling Laws in Two-Layer Networks with Power-Law Data Spectra

Neural scaling laws describe how the performance of deep neural networks scales with key factors such as training data size, model complexity, and training time, often following power-law behaviors over multiple orders of magnitude. Despite their empirical observation, the theoretical understanding…