Mansi Dhanania

How do Transformer Embeddings Represent Compositions? A Functional Analysis

Compositionality is a key aspect of human intelligence, essential for reasoning and generalization. While transformer-based models have become the de facto standard for many language modeling tasks, little is known about how they represent compound words, and whether these representations are compos…