Huchuan Lu*

SHERL: Synthesizing High Accuracy and Efficient Memory for Resource-Limited Transfer Learning

"Parameter-efficient transfer learning (PETL) has emerged as a flourishing research field for adapting large pre-trained models to downstream tasks, greatly reducing trainable parameters while grappling with memory challenges during fine-tuning. To address it, memory-efficient series (METL) avoid ba…