Speech Collage: Code-Switched Audio Generation by Collaging Monolingual Corpora
Amir Hussein, Dorsa Zeinali, Ondrej Klejch, Matthew Wiesner, Brian Yan, Shammur Absar Chowdhury, Ahmed Ali, Shinji Watanabe
Abstract
Designing effective automatic speech recognition (ASR) systems for Code-Switching (CS) often depends on the availability of the transcribed CS resources. To address data scarcity, this paper introduces Speech Collage, a method that synthesizes CS data from monolingual corpora by splicing audio segments. We further improve the smoothness quality of audio generation using an overlap-add approach. We investigate the impact of generated data on speech recognition in two scenarios: using in-domain CS text and a zero-shot approach with synthesized CS text. Empirical results highlight up to 34.4% and 16.2% relative reductions in Mixed-Error Rate and Word-Error Rate for in-domain and zero-shot scenarios, respectively. Lastly, we demonstrate that CS augmentation bolsters the model’s code-switching inclination and reduces its monolingual bias.
BibTeX
@inproceedings{icassp2024_speechcollagecod,
title = {Speech Collage: Code-Switched Audio Generation by Collaging Monolingual Corpora},
author = {Amir Hussein and Dorsa Zeinali and Ondrej Klejch and Matthew Wiesner and Brian Yan and Shammur Absar Chowdhury and Ahmed Ali and Shinji Watanabe and Sanjeev Khudanpur},
booktitle = {ICASSP 2024},
year = {2024}
}