BOUQuET : dataset, Benchmark and Open initiative for Universal Quality Evaluation in Translation
Pierre Andrews, Mikel Artetxe, Mariano Coria Meglioli, Marta R. Costa-juss{\`a}, Joe Chuang, David Dale, Mark Duppenthaler, Nathanial Paul Ekberg
Abstract
BOUQuET is a multi-way, multicentric and multi-register/domain dataset and benchmark, and a broader collaborative initiative. This dataset is handcrafted in 8 non-English languages (i.e. Egyptian Arabic and Modern Standard Arabic, French, German, Hindi, Indonesian, Mandarin Chinese, Russian, and Spanish). Each of these source languages are representative of the most widely spoken ones and therefore they have the potential to serve as pivot languages that will enable more accurate translations. The dataset is multicentric to enforce representation of multilingual language features. In addition, the dataset goes beyond the sentence level, as it is organized in paragraphs of various lengths. Compared with related machine translation datasets, we show that BOUQuET has a broader representation of domains while simplifying the translation task for non-experts. Therefore, BOUQuET is specially suitable for crowd-source extension for which we are launching a call aim-ing at collecting a multi-way parallel corpus covering any written language. The dataset is freely available at https://huggingface.co/datasets/facebook/bouquet.
BibTeX
@inproceedings{emnlp2025_bouquetdatasetbe,
title = {BOUQuET : dataset, Benchmark and Open initiative for Universal Quality Evaluation in Translation},
author = {Pierre Andrews and Mikel Artetxe and Mariano Coria Meglioli and Marta R. Costa-juss{\`a} and Joe Chuang and David Dale and Mark Duppenthaler and Nathanial Paul Ekberg and Cynthia Gao and Daniel Edward Licht and Jean Maillard and Alexandre Mourachko and Christophe Ropers and Safiyyah Saleem and Eduardo S{\'a}nchez and Ioannis Tsiamas and Arina Turkatenko and Albert Ventayol-Boada and Shireen Yates},
booktitle = {EMNLP 2025},
year = {2025}
}