Sabarish Sainathan — accepted AI-conference papers

2023

Feature-Learning Networks Are Consistent Across Widths At Realistic Scales

Nikhil Vyas, Alexander Atanasov, Blake Bordelon, Depen Morwani, Sabarish Sainathan, Cengiz Pehlevan

NeurIPS 2023poster

We study the effect of width on the dynamics of feature-learning neural networks across a variety of architectures and datasets. Early in training, wide neural networks trained on online data have not only identical loss curves but also agree in their point-wise test predictions throughout training.…

Cited by 31 Source ↗PDF ↗Scholar ↗

2023

The Onset of Variance-Limited Behavior for Networks in the Lazy and Rich Regimes

Alexander Atanasov, Blake Bordelon, Sabarish Sainathan, Cengiz Pehlevan

ICLR 2023poster

For small training set sizes $P$, the generalization error of wide neural networks is well-approximated by the error of an infinite width neural network (NN), either in the kernel or mean-field/feature-learning regime. However, after a critical sample size $P^*$, we empirically find the finite-width…

Cited by 32 Source ↗PDF ↗code ↗Scholar ↗