La Leaderboard: A Large Language Model Leaderboard for Spanish Varieties and Languages of Spain and Latin America
María Grandury, Javier Aula-Blasco, Júlia Falcão, Clémentine Fourrier, Miguel González Saiz, Gonzalo Martínez, Gonzalo Santamaria Gomez, Rodrigo Agerri
Abstract
Leaderboards showcase the current capabilities and limitations of Large Language Models (LLMs). To motivate the development of LLMs that represent the linguistic and cultural diversity of the Spanish-speaking community, we present La Leaderboard, the first open-source leaderboard to evaluate generative LLMs in languages and language varieties of Spain and Latin America. La Leaderboard is a community-driven project that aims to establish an evaluation standard for everyone interested in developing LLMs for the Spanish-speaking community. This initial version combines 66 datasets in Catalan, Basque, Galician, and different Spanish varieties, showcasing the evaluation results of 50 models. To encourage community-driven development of leaderboards in other languages, we explain our methodology, including guidance on selecting the most suitable evaluation setup for each downstream task. In particular, we provide a rationale for using fewer few-shot examples than typically found in the literature, aiming to reduce environmental impact and facilitate access to reproducible results for a broader research community.
BibTeX
@inproceedings{grandury-etal-2025-la,
title = "La Leaderboard: A Large Language Model Leaderboard for {S}panish Varieties and Languages of {S}pain and {L}atin {A}merica",
author = "Grandury, Mar{\'i}a and
Aula-Blasco, Javier and
Falc{\~a}o, J{\'u}lia and
Fourrier, Cl{\'e}mentine and
Saiz, Miguel Gonz{\'a}lez and
Mart{\'i}nez, Gonzalo and
Gomez, Gonzalo Santamaria and
Agerri, Rodrigo and
Garc{\'i}a, Nuria Aldama and
Chiruzzo, Luis and
Conde, Javier and
Gomez Adorno, Helena and
Nieto, Marta Guerrero and
Ivetta, Guido and
Fuertes, Nat{\`a}lia L{\'o}pez and
Plaza-del-Arco, Flor Miriam and
Mart{\'i}n-Valdivia, Mar{\'i}a-Teresa and
Zamorano, Helena Montoro and
Sanz, Carmen Mu{\~n}oz and
Reviriego, Pedro and
Plaza, Leire Rosado and
Vaca Serrano, Alejandro and
Vallecillo-Rodr{\'i}guez, Estrella and
Vallego, Jorge and
Zubiaga, Irune",
editor = "Che, Wanxiang and
Nabende, Joyce and
Shutova, Ekaterina and
Pilehvar, Mohammad Taher",
booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
month = jul,
year = "2025",
address = "Vienna, Austria",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.acl-long.1561/",
doi = "10.18653/v1/2025.acl-long.1561",
pages = "32482--32524",
ISBN = "979-8-89176-251-0"
}