Annealing Distillation Algorithm for Transferring Unsupervised Clustering Knowledge to Supervised Student Models
In knowledge distillation, the performance of teacher models often serves as an upper limit for student models. For a long time, deeper and more accurate supervised learning algorithms have been the first choice for teacher models in image classification tasks where unsupervised models typically und…