2026
Theoretical Characterization of Generalization in Knowledge Distillation
ICML 2026poster
This paper presents a theoretical investigation into the generalization capabilities of cross-domain knowledge distillation. Utilizing a high-dimensional asymptotic analysis of a linear teacher–student model, we characterize the excess risk while accounting for both model and covariate shifts. Our r…