ACL 2025finding0 citations
2M-BELEBELE: Highly Multilingual Speech and American Sign Language Comprehension Dataset Download PDF
Marta R. Costa-jussà, Bokai Yu, Pierre Andrews, Belen Alastruey, Necati Cihan Camgoz, Joe Chuang, Jean Maillard, Christophe Ropers
Abstract
We introduce the first highly multilingual speech and American Sign Language (ASL) comprehension dataset by extending BELEBELE. Our dataset covers 91 spoken languages at the intersection of BELEBELE and FLEURS, and one sign language (ASL). As a by-product we also extend the Automatic Speech Recognition Benchmark, FLEURS, by 20%. We evaluate 2M-BELEBELE dataset for both 5-shot and zero-shot settings and across languages, the speech comprehension accuracy is ≈ 10% average lower compared to reading comprehension.
BibTeX
@inproceedings{costa-jussa-etal-2025-2m,
title = "2{M}-{BELEBELE}: Highly Multilingual Speech and {A}merican {S}ign {L}anguage Comprehension Dataset Download {PDF}",
author = "Costa-juss{\`a}, Marta R. and
Yu, Bokai and
Andrews, Pierre and
Alastruey, Belen and
Camgoz, Necati Cihan and
Chuang, Joe and
Maillard, Jean and
Ropers, Christophe and
Turkatenko, Arina and
Wood, Carleigh",
editor = "Che, Wanxiang and
Nabende, Joyce and
Shutova, Ekaterina and
Pilehvar, Mohammad Taher",
booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
month = jul,
year = "2025",
address = "Vienna, Austria",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.findings-acl.569/",
doi = "10.18653/v1/2025.findings-acl.569",
pages = "10893--10904",
ISBN = "979-8-89176-256-5"
}