Challenging America: Modeling language in longer time scales
Jakub Pokrywka, Filip Graliński, Krzysztof Jassem, Karol Kaczmarek, Krzysztof Jurkiewicz, Piotr Wierzchon
Abstract
The aim of the paper is to apply, for historical texts, the methodology used commonly to solve various NLP tasks defined for contemporary data, i.e. pre-train and fine-tune large Transformer models. This paper introduces an ML challenge, named Challenging America (ChallAm), based on OCR-ed excerpts from historical newspapers collected from the Chronicling America portal. ChallAm provides a dataset of clippings, labeled with metadata on their origin, and paired with their textual contents retrieved by an OCR tool. Three, publicly available, ML tasks are defined in the challenge: to determine the article date, to detect the location of the issue, and to deduce a word in a text gap (cloze test). Strong baselines are provided for all three ChallAm tasks. In particular, we pre-trained a RoBERTa model from scratch from the historical texts. We also discuss the issues of discrimination and hate-speech present in the historical American texts.
BibTeX
@inproceedings{pokrywka-etal-2022-challenging,
title = "Challenging {A}merica: Modeling language in longer time scales",
author = "Pokrywka, Jakub and
Grali{\'n}ski, Filip and
Jassem, Krzysztof and
Kaczmarek, Karol and
Jurkiewicz, Krzysztof and
Wierzchon, Piotr",
editor = "Carpuat, Marine and
de Marneffe, Marie-Catherine and
Meza Ruiz, Ivan Vladimir",
booktitle = "Findings of the Association for Computational Linguistics: NAACL 2022",
month = jul,
year = "2022",
address = "Seattle, United States",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.findings-naacl.56/",
doi = "10.18653/v1/2022.findings-naacl.56",
pages = "737--749"
}