2023
TokenDrop + BucketSampler: Towards Efficient Padding-free Fine-tuning of Language Models
EMNLP 2023long findings
The great success of Language Models (LMs) for various Natural Language Processing (NLP) tasks is accompanied by computational challenges during both pre-training and fine-tuning. Pre-training has attracted significant attention due to its huge computational footprint. We focus on the fine-tuning of…