Jonas Scheunemann

Spectral Reach: Understanding Neural Scaling through Kernel Alignment Dynamics

Neural scaling laws describe predictable power-law relationships between model size, dataset size, compute cost, and performance. While these laws are applied to improve the performance of modern foundation models, the mechanisms underpinning them are less understood, in part due to the absence of s…