2024
SlimFit: Memory-Efficient Fine-Tuning of Transformer-based Models Using Training Dynamics
NAACL 2024long
Transformer-based models, such as BERT and ViT, have achieved state-of-the-art results across different natural language processing (NLP) and computer vision (CV) tasks. However, these models are extremely memory intensive during their fine-tuning process, making them difficult to deploy on GPUs wit…