Lehan Chen

Improved Scaling Laws via Weak-to-Strong Generalization in Random Features Ridge Regression

It is increasingly common in machine learning to use learned models to label data and then employ such data to train more capable models. The phenomenon of weak-to-strong generalization exemplifies the advantage of this two-stage procedure: a strong student is trained on imperfect labels obtained fr…