Evaluation Dataset for Lexical Translation Consistency in Chinese-to-English Document-level Translation
Xiangyu Lei, Junhui Li, Shimin Tao, Hao Yang
Abstract
Lexical translation consistency is one of the most common discourse phenomena in Chinese-to-English document-level translation. To better evaluate the performance of lexical translation consistency, previous researches assumes that all repeated source words should be translated consistently. However, constraining translations of repeated source words to be consistent will hurt word diversity and human translators tend to use different words in translation. Therefore, in this paper we construct a test set of 310 bilingual news articles to properly evaluate lexical translation consistency. We manually differentiate those repeated source words whose translations are consistent into two types: true consistency and false consistency. Then based on the constructed test set, we evaluate the performance of lexical translation consistency for several typical NMT systems.
BibTeX
@inproceedings{lei-etal-2024-evaluation,
title = "Evaluation Dataset for Lexical Translation Consistency in {C}hinese-to-{E}nglish Document-level Translation",
author = "Lei, Xiangyu and
Li, Junhui and
Tao, Shimin and
Yang, Hao",
editor = "Calzolari, Nicoletta and
Kan, Min-Yen and
Hoste, Veronique and
Lenci, Alessandro and
Sakti, Sakriani and
Xue, Nianwen",
booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
month = may,
year = "2024",
address = "Torino, Italia",
publisher = "ELRA and ICCL",
url = "https://aclanthology.org/2024.lrec-main.583/",
pages = "6575--6581"
}