2023
Adaptive Knowledge Distillation Between Text and Speech Pre-Trained Models
ICASSP 2023accepted
Learning on a massive amount of speech corpus leads to the recent success of many self-supervised speech models. With knowledge distillation, these models may also benefit from the knowledge encoded by language models that are pre-trained on rich sources of texts. The distillation process, however,…