2023
Not Enough Data to Pre-train Your Language Model? MT to the Rescue!
ACL 2023findings
In recent years, pre-trained transformer-based language models (LM) have become a key resource for implementing most NLP tasks. However, pre-training such models demands large text collections not available in most languages. In this paper, we study the use of machine-translated corpora for pre-trai…