Emanuele Moscato

How Transformers Learn Structured Data: Insights From Hierarchical Filtering

Understanding the learning process and the embedded computation in transformers is becoming a central goal for the development of interpretable AI. In the present study, we introduce a hierarchical filtering procedure for data models of sequences on trees, allowing us to hand-tune the range of posit…