Extending Multilingual ASR to New Languages Using Supplementary Encoder and Decoder Components

Yerbolat Khassanov; Zhipeng Chen; Tianfeng Chen; Tze Yuang Chong; Wei Li; Lu Lu; Zejun Ma

ICASSP 2024accepted0 citations

Extending Multilingual ASR to New Languages Using Supplementary Encoder and Decoder Components

Yerbolat Khassanov, Zhipeng Chen, Tianfeng Chen, Tze Yuang Chong, Wei Li, Lu Lu, Zejun Ma

Abstract

Extending multilingual automatic speech recognition (mASR) systems to new languages poses challenges, particularly when training data for existing languages is limited or unavailable. To tackle this issue, we suggest utilizing supplementary encoder and decoder components. Specifically, we propose appending and fine-tuning a distinct decoder designed for new languages, while preserving the parameters of existing languages to minimize disruption to their performance. Furthermore, we advocate attaching an additional encoder component to enhance acoustic representation learning for new languages, resulting in substantial improvements in word error rate performance. Our experimental findings demonstrate the effectiveness of the proposed methods for the task of extending language support within mASR systems.

BibTeX

@inproceedings{icassp2024_extendingmultili,
  title = {Extending Multilingual ASR to New Languages Using Supplementary Encoder and Decoder Components},
  author = {Yerbolat Khassanov and Zhipeng Chen and Tianfeng Chen and Tze Yuang Chong and Wei Li and Lu Lu and Zejun Ma},
  booktitle = {ICASSP 2024},
  year = {2024}
}