Rita Brugarolas Brufau

Token and Head Adaptive Transformers for Efficient Natural Language Processing

While pre-trained language models like BERT have achieved impressive results on various natural language processing tasks, deploying them on resource-restricted devices is challenging due to their intensive computational cost and memory footprint. Previous approaches mainly focused on training small…