Archer Yang

When Iteration Helps and Hurts in Self-Training: Denoising vs. Signal Forgetting

Iterative self-training (self-distillation) repeatedly refits a model on pseudo-labels generated by its own predictions. We study this procedure in overparameterized linear regression: an initial estimator is trained on noisy labels, and each subsequent iterate is trained on fresh covariates with no…