Joint Training and Decoding for Multilingual End-to-End Simultaneous Speech Translation
Wuwei Huang, Renren Jin, Wen Zhang, Jian Luan, Bin Wang, Deyi Xiong
Abstract
Recent studies on end-to-end speech translation(ST) have facilitated the exploration of multilingual end-to-end ST and end-to-end simultaneous ST. In this paper, we investigate end-to-end simultaneous speech translation in a one-to-many multilingual setting which is closer to applications in real scenarios. We explore a separate decoder architecture and a unified architecture for joint synchronous training in this scenario. To further explore knowledge transfer across languages, we propose an asynchronous training strategy on the proposed unified decoder architecture. A multi-way aligned multilingual end-to-end ST dataset was curated as a benchmark testbed to evaluate our methods. Experimental results demonstrate the effectiveness of our models on the collected dataset. Our codes and data are available at: https://github.com/XiaoMi/TED-MMST.
BibTeX
@inproceedings{icassp2023_jointtrainingand,
title = {Joint Training and Decoding for Multilingual End-to-End Simultaneous Speech Translation},
author = {Wuwei Huang and Renren Jin and Wen Zhang and Jian Luan and Bin Wang and Deyi Xiong},
booktitle = {ICASSP 2023},
year = {2023}
}