Hyunjune Shin

Teacher as a Lenient Expert: Teacher-Agnostic Data-Free Knowledge Distillation

Data-free knowledge distillation (DFKD) aims to distill pretrained knowledge to a student model with the help of a generator without using original data. In such data-free scenarios, achieving stable performance of DFKD is essential due to the unavailability of validation data. Unfortunately, this p…