Yohai Bar-Sinai

2 accepted papers

2025

Grokking at the Edge of Linear Separability

Alon Beck, Noam Itzhak Levi, Yohai Bar-Sinai

ICML 2025poster

We investigate the phenomenon of grokking -- delayed generalization accompanied by non-monotonic test loss behavior -- in a simple binary logistic classification task, for which "memorizing" and "generalizing" solutions can be strictly defined. Surprisingly, we find that grokking arises naturally e…

Cited by 0 Source ↗PDF ↗Scholar ↗

2024

Grokking in Linear Estimators -- A Solvable Model that Groks without Understanding

Noam Itzhak Levi, Alon Beck, Yohai Bar-Sinai

ICLR 2024poster

Grokking is the intriguing phenomenon where a model learns to generalize long after it has fit the training data. We show both analytically and numerically that grokking can surprisingly occur in linear networks performing linear tasks in a simple teacher-student setup. In this setting, the ful…

Cited by 9 Source ↗PDF ↗Scholar ↗