Daniel Vila-Suero

Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation

Reliable multilingual evaluation is difficult, and culturally appropriate evaluation is even harder to achieve.A common practice to fill this gap is to machine-translate English evaluation sets. However, translation introduces language bias and carries over cultural and regional assumptions from the…