Aviv Ben-Arie

What’s in Your Head? Emergent Behaviour in Multi-Task Transformer Models

The primary paradigm for multi-task training in natural language processing is to represent the input with a shared pre-trained language model, and add a small, thin network (head) per task. Given an input, a target head is the head that is selected for outputting the final prediction. In this work,…