Empirical Regularization for Synthetic Sentence Pairs in Unsupervised Neural Machine Translation
Abstract
UNMT tackles translation on monolingual corpora in two required languages. Since there is no explicitly cross-lingual signal, pre-training and synthetic sentence pairs are significant to the success of UNMT. In this work, we empirically study the core training procedure of UNMT to analyze the synthetic sentence pairs obtained from back-translation. We introduce new losses to UNMT to regularize the synthetic sentence pairs by jointly training the UNMT objective and the regularization objective. Our comprehensive experiments support that our method can generally improve the performance of currently successful models on three similar pairs {French, German, Romanian} English and one dissimilar pair Russian English with acceptably additional cost.
BibTeX
@inproceedings{aaai2021_empiricalregular,
title = {Empirical Regularization for Synthetic Sentence Pairs in Unsupervised Neural Machine Translation},
author = {Xi Ai and Bin Fang},
booktitle = {AAAI 2021},
year = {2021}
}