Tamara Czinczoll — accepted AI-conference papers

2024

NextLevelBERT: Masked Language Modeling with Higher-Level Representations for Long Documents

Tamara Czinczoll, Christoph Hönes, Maximilian Schall, Gerard De Melo

ACL 2024long

While (large) language models have significantly improved over the last years, they still struggle to sensibly process long sequences found, e.g., in books, due to the quadratic scaling of the underlying attention mechanism. To address this, we propose NextLevelBERT, a Masked Language Model operatin…

Cited by 4 Source ↗PDF ↗code ↗Scholar ↗

2022

Scientific and Creative Analogies in Pretrained Language Models

Tamara Czinczoll, Helen Yannakoudakis, Pushkar Mishra, Ekaterina Shutova

EMNLP 2022finding

This paper examines the encoding of analogy in large-scale pretrained language models, such as BERT and GPT-2. Existing analogy datasets typically focus on a limited set of analogical relations, with a high similarity of the two domains between which the analogy holds. As a more realistic setup, we…

Cited by 27 Source ↗PDF ↗code ↗Scholar ↗