2025
Can Out-of-Distribution Evaluations Uncover Reliance on Prediction Shortcuts? A Case Study in Question Answering
Michal {\v{S}}tef{\'a}nik, Timothee Mickus, Michal Spiegel, Marek Kadl{\v{c}}{\'i}k, Josef Kucha{\v{r}}
EMNLP 2025
A large body of recent work assesses models’ generalization capabilities through the lens of performance on out-of-distribution (OOD) datasets. Despite their practicality, such evaluations build upon a strong assumption: that OOD evaluations can capture and reflect upon possible failures in a real-w