Injecting Text and Cross-Lingual Supervision in Few-Shot Learning from Self-Supervised Models

Matthew Wiesner; Desh Raj; Sanjeev Khudanpur

ICASSP 2022accepted0 citations

Injecting Text and Cross-Lingual Supervision in Few-Shot Learning from Self-Supervised Models

Matthew Wiesner, Desh Raj, Sanjeev Khudanpur

Abstract

Self-supervised model pretraining has recently garnered significant interest. However, using additional resources in fine-tuning these models has received less attention. We demonstrate how universal phoneset acoustic models can leverage cross-lingual supervision to improve transfer of pretrained self-supervised representations to new languages. We also show how target-language text can be used to enable and improve fine-tuning with the lattice-free maximum mutual information (LF-MMI) objective. In three low-resource languages these techniques greatly improved few-shot learning performance.

BibTeX

@inproceedings{icassp2022_injectingtextand,
  title = {Injecting Text and Cross-Lingual Supervision in Few-Shot Learning from Self-Supervised Models},
  author = {Matthew Wiesner and Desh Raj and Sanjeev Khudanpur},
  booktitle = {ICASSP 2022},
  year = {2022}
}