Jerry Song

From Linear to Nonlinear: Provable Weak-to-Strong Generalization through Feature Learning

Weak-to-strong generalization refers to the phenomenon where a stronger model trained under supervision from a weaker one can outperform its teacher. While prior studies aim to explain this effect, most theoretical insights are limited to abstract frameworks or linear/random feature models. In this…