Reeka Estacio

Not quite Sherlock Holmes: Language model predictions do not reliably differentiate impossible from improbable events

Can language models reliably predict that possible events are more likely than merely improbable ones? By teasing apart possibility, typicality, and contextual relatedness, we show that despite the results of previous work, language models’ ability to do this is far from robust. In fact, under certa…