AIConfPaper
← Search

Isaac Rayburn Caswell

6 accepted papers

2026

ATLAS: Adaptive Transfer Scaling Laws for Multilingual Pretraining, Finetuning, and Decoding the Curse of Multilinguality

Shayne Longpre, Sneha Kudugunta, Niklas Muennighoff, I-Hung Hsu, Isaac Rayburn Caswell, Alex Pentland +3
ICLR 2026poster

Scaling laws research has focused overwhelmingly on English—yet the most prominent AI models explicitly serve billions of international users. In this work, we undertake the largest multilingual scaling laws study to date, totaling 774 multilingual training experiments, spanning 10M-8B model paramet…

Cited by 0Source ↗Scholar ↗
2025

Alligators All Around: Mitigating Lexical Confusion in Low-resource Machine Translation

Elizabeth Nielsen, Isaac Rayburn Caswell, Jiaming Luo, Colin Cherry
NAACL 2025short

Current machine translation (MT) systems for low-resource languages have a particular failure mode: When translating words in a given domain, they tend to confuse words within that domain. So, for example, “lion” might be translated as “alligator”, and “orange” might be rendered as “purple.” We prop…

Cited by 1Source ↗PDF ↗Scholar ↗
2025

WMT24++: Expanding the Language Coverage of WMT24 to 55 Languages & Dialects

Daniel Deutsch, Eleftheria Briakou, Isaac Rayburn Caswell, Mara Finkelstein, Rebecca Galor, Juraj Juraska +11
ACL 2025finding

As large language models (LLM) become more and more capable in languages other than English, it is important to collect benchmark datasets in order to evaluate their multilingual performance, including on tasks like machine translation (MT). In this work, we extend the WMT24 dataset to cover 55 lang…

Cited by 0Source ↗PDF ↗Scholar ↗
2023

GATITOS: Using a New Multilingual Lexicon for Low-resource Machine Translation

Alexander Jones, Isaac Rayburn Caswell, Orhan Firat, Ishank Saxena
EMNLP 2023long main

Modern machine translation models and language models are able to translate without having been trained on parallel data, greatly expanding the set of languages that they can serve. However, these models still struggle in a variety of predictable ways, a problem that cannot be overcome without at l…

Cited by 0Source ↗Scholar ↗
2023

MADLAD-400: A Multilingual And Document-Level Large Audited Dataset

Sneha Kudugunta, Isaac Rayburn Caswell, Biao Zhang, Xavier Garcia, Derrick Xin, Aditya Kusupati +3
NeurIPS 2023poster

We introduce MADLAD-400, a manually audited, general domain 3T token monolingual dataset based on CommonCrawl, spanning 419 languages. We discuss the limitations revealed by self-auditing MADLAD-400, and the role data auditing had in the dataset creation process. We then train and release a 10.7B-pa…

Cited by 126Source ↗PDF ↗Scholar ↗
2023

XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languages

Sebastian Ruder, Jonathan H. Clark, Alexander Gutkin, Mihir Kale, Min Ma, Massimo Nicosia +21
EMNLP 2023long findings

Data scarcity is a crucial issue for the development of highly multilingual NLP systems. Yet for many under-represented languages (ULs) --- languages for which NLP research is particularly far behind in meeting user needs --- it is feasible to annotate small amounts of data. Motivated by this, we pr…

Cited by 0Source ↗code ↗Scholar ↗