2023
Better Teacher Better Student: Dynamic Prior Knowledge for Knowledge Distillation
ICLR 2023poster
Knowledge distillation (KD) has shown very promising capabilities in transferring learning representations from large models (teachers) to small models (students). However, as the capacity gap between students and teachers becomes larger, existing KD methods fail to achieve better results. Our work…