2022
MAKD: MULTIPLE Auxiliary Knowledge Distillation
ICASSP 2022accepted
Knowledge distillation aims to learn a small student model by leveraging knowledge from a larger teacher model. The gap between these heterogeneous models hinder their knowledge transfer and it would be more challenging when the teacher model is from another task. Previous methods view the teacher m…