Giorgi Kvernadze

BERT & Family Eat Word Salad: Experiments with Text Understanding

In this paper, we study the response of large models from the BERT family to incoherent inputs that should confuse any model that claims to understand natural language. We define simple heuristics to construct such examples. Our experiments show that state-of-the-art models consistently fail to reco…