Théo Desbordes

Can Transformers Process Recursive Nested Constructions, Like Humans?

Recursive processing is considered a hallmark of human linguistic abilities. A recent study evaluated recursive processing in recurrent neural language models (RNN-LMs) and showed that such models perform below chance level on embedded dependencies within nested constructions – a prototypical exampl…