Joint Unsupervised and Supervised Training for Automatic Speech Recognition via Bilevel Optimization
A. F. M. Saif, Xiaodong Cui, Han Shen, Songtao Lu, Brian Kingsbury, Tianyi Chen
Abstract
In this paper, we present a novel bilevel optimization-based training approach to training acoustic models for automatic speech recognition (ASR) tasks that we term bi-level joint unsupervised and supervised training (BL-JUST). BL-JUST employs a lower and upper level optimization with an unsupervised loss and a supervised loss respectively, leveraging recent advances in penalty-based bilevel optimization to solve this challenging ASR problem with affordable complexity and rigorous convergence guarantees. To evaluate BL-JUST, extensive experiments on the LibriSpeech and TED-LIUM v2 datasets have been conducted. BL-JUST achieves superior performance over the commonly used pre-training followed by fine-tuning strategy.
BibTeX
@inproceedings{icassp2024_jointunsupervise,
title = {Joint Unsupervised and Supervised Training for Automatic Speech Recognition via Bilevel Optimization},
author = {A. F. M. Saif and Xiaodong Cui and Han Shen and Songtao Lu and Brian Kingsbury and Tianyi Chen},
booktitle = {ICASSP 2024},
year = {2024}
}