Jacek Wiland — accepted AI-conference papers

2025

LM-Pub-Quiz: A Comprehensive Framework for Zero-Shot Evaluation of Relational Knowledge in Language Models

Max Ploner, Jacek Wiland, Sebastian Pohl, Alan Akbik

NAACL 2025system demonstrations

Knowledge probing evaluates to which extent a language model (LM) has acquired relational knowledge during its pre-training phase. It provides a cost-effective means of comparing LMs of different sizes and training setups and is useful for monitoring knowledge gained or lost during continual learnin…

Cited by 0 Source ↗PDF ↗code ↗Scholar ↗

2024

BEAR: A Unified Framework for Evaluating Relational Knowledge in Causal and Masked Language Models

Jacek Wiland, Max Ploner, Alan Akbik

NAACL 2024findings

Knowledge probing assesses to which degree a language model (LM) has successfully learned relational knowledge during pre-training. Probing is an inexpensive way to compare LMs of different sizes and training configurations. However, previous approaches rely on the objective function used in pre-tra…

Cited by 3 Source ↗PDF ↗code ↗Scholar ↗