Lukas Garbaciauskas

Choose Your Transformer: Improved Transferability Estimation of Transformer Models on Classification Tasks

There currently exists a multitude of pre-trained transformer language models (LMs) that are readily available. From a practical perspective, this raises the question of which pre-trained LM will perform best if fine-tuned for a specific downstream NLP task. However, exhaustively fine-tuning all ava…