2025
Rethinking Self-Distillation: Label Averaging and Enhanced Soft Label Refinement with Partial Labels
ICLR 2025poster
We investigate the mechanisms of self-distillation in multi-class classification, particularly in the context of linear probing with fixed feature extractors where traditional feature learning explanations do not apply. Our theoretical analysis reveals that multi-round self-distillation effectively…