Improving Kinyarwanda Speech Recognition Via Semi-Supervised Learning

Abstract

Achieving robust speech recognition for Kinyarwanda remains a challenging task. In this work, we empirically show that using self-supervised pretraining, following a curriculum schedule during fine-tuning and using semi-supervised learning improve speech recognition for Kinyarwanda. Our approach focuses on using public domain data only. A new studio-quality speech dataset is collected from a public website, aligned to text and then used to formulate a simple curriculum learning schedule for training on a larger, noisier dataset. After four generations of semi-supervised learning, our final model achieves 3.2% word error rate (WER) on the new dataset and 15.9% WER on Mozilla Common Voice benchmark. These results improve upon off-the-shelf models that use English language self-supervised representations. Our experiments also indicate that using syllabic rather than character-based tokenization results in better speech recognition for Kinyarwanda.