Improving the Quality Trade-Off for Neural Machine Translation Multi-Domain Adaptation
Eva Hasler, Tobias Domhan, Jonay Trenous, Ke Tran, Bill Byrne, Felix Hieber
Abstract
Building neural machine translation systems to perform well on a specific target domain is a well-studied problem. Optimizing system performance for multiple, diverse target domains however remains a challenge. We study this problem in an adaptation setting where the goal is to preserve the existing system quality while incorporating data for domains that were not the focus of the original translation system. We find that we can improve over the performance trade-off offered by Elastic Weight Consolidation with a relatively simple data mixing strategy. At comparable performance on the new domains, catastrophic forgetting is mitigated significantly on strong WMT baselines. Combining both approaches improves the Pareto frontier on this task.
BibTeX
@inproceedings{hasler-etal-2021-improving,
title = "Improving the Quality Trade-Off for Neural Machine Translation Multi-Domain Adaptation",
author = "Hasler, Eva and
Domhan, Tobias and
Trenous, Jonay and
Tran, Ke and
Byrne, Bill and
Hieber, Felix",
editor = "Moens, Marie-Francine and
Huang, Xuanjing and
Specia, Lucia and
Yih, Scott Wen-tau",
booktitle = "Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing",
month = nov,
year = "2021",
address = "Online and Punta Cana, Dominican Republic",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2021.emnlp-main.666/",
doi = "10.18653/v1/2021.emnlp-main.666",
pages = "8470--8477"
}