DiscoX: Benchmarking Discourse-Level Translation in Expert Domains
Xiying ZHAO, Zhoufutu Wen, Zhixuan Chen, Jingzhe Ding, Jianpeng Jiao, Shuai Li, Xi Li, Danni Liang
Abstract
The evaluation of discourse-level translation in expert domains remains inadequate, despite its centrality to knowledge dissemination and cross-lingual scholarly communication. While these translations demand discourse-level coherence and strict terminological precision, current evaluation methods predominantly focus on segment-level accuracy and fluency. To address this limitation, we introduce DiscoX, a new benchmark for discourse-level and expert-level Chinese-English translation. It comprises 200 professionally-curated texts from 7 domains, with an average length exceeding 1700 tokens. To evaluate performance on DiscoX, we also develop Metric-S, a reference-free system that provides fine-grained automatic assessments across accuracy, fluency, and appropriateness. Metric-S demonstrates strong consistency with human judgments, significantly outperforming existing metrics. Our experiments reveal a remarkable performance gap: even the most advanced LLMs still trail human experts on these tasks. This finding validates the difficulty of DiscoX and underscores the challenges that remain in achieving professional-grade machine translation. The proposed benchmark and evaluation system provide a robust framework for more rigorous evaluation, facilitating future advancements in LLM-based translation. Our data and code are available at https://anonymous.4open.science/r/DiscoX-5F18.
BibTeX
@inproceedings{
zhao2026discox,
title={DiscoX: Benchmarking Discourse-Level Translation in Expert Domains},
author={Xiying ZHAO and Zhoufutu Wen and Zhixuan Chen and Jingzhe Ding and Jianpeng Jiao and Shuai Li and Xi Li and Danni Liang and Shengda Long and Qianqian Liu and Xianbo Wu and Hongwan Gao and Xiang Gao and LIANG HU and Jiashuo Liu and Liumengyun and Weiran Shi and Chenghao Yang and Qianyu Yang and Xuanliang Zhang and Ge Zhang and Wenhao Huang},
booktitle={The Fourteenth International Conference on Learning Representations},
year={2026},
url={https://openreview.net/forum?id=OTCfZ6h8Pe}
}