Anna Ollerenshaw

Towards Domain Generalisation in ASR with Elitist Sampling and Ensemble Knowledge Distillation

Knowledge distillation (KD) has widely been used for model compression and domain adaptation for speech applications. In the presence of multiple teachers, knowledge can easily be transferred to the student by averaging the models output. However, previous research shows that the student do not adap…