Roberto Lotufo — accepted AI-conference papers

2024

Measuring Cross-lingual Transfer in Bytes

Leandro De Souza, Thales Almeida, Roberto Lotufo, Rodrigo Frassetto Nogueira

NAACL 2024long

Multilingual pretraining has been a successful solution to the challenges posed by the lack of resources for languages. These models can transfer knowledge to target languages with minimal or no examples. Recent research suggests that monolingual models also have a similar capability, but the mechan…

Cited by 2 Source ↗PDF ↗code ↗Scholar ↗

2022

MonoByte: A Pool of Monolingual Byte-level Language Models

Hugo Abonizio, Leandro Rodrigues de Souza, Roberto Lotufo, Rodrigo Nogueira

COLING 2022main

The zero-shot cross-lingual ability of models pretrained on multilingual and even monolingual corpora has spurred many hypotheses to explain this intriguing empirical result. However, due to the costs of pretraining, most research uses public models whose pretraining methodology, such as the choice…

Cited by 1 Source ↗PDF ↗code ↗Scholar ↗