Enhancing Unsupervised Speech Recognition with Diffusion GANS

Xianchao Wu

ICASSP 2023accepted0 citations

Enhancing Unsupervised Speech Recognition with Diffusion GANS

Abstract

We enhance the vanilla adversarial training method for unsupervised Automatic Speech Recognition (ASR) by a diffusionGAN. Our model (1) injects instance noises of various intensities to the generator’s output and unlabeled reference text which are sampled from pretrained phoneme language models with a length constraint, (2) asks diffusion timestep-dependent discriminators to separate them, and (3) back-propagates the gradients to update the generator. Word/phoneme error rate comparisons with wav2vec-U under Librispeech (3.1% for test-clean and 5.6% for test-other), TIMIT and MLS datasets, show that our enhancement strategies work effectively.

BibTeX

@inproceedings{icassp2023_enhancingunsuper,
  title = {Enhancing Unsupervised Speech Recognition with Diffusion GANS},
  author = {Xianchao Wu},
  booktitle = {ICASSP 2023},
  year = {2023}
}