Yuesong Zhang

PYRA: Parallel Yielding Re-Activation for Training-Inference Efficient Task Adaptation

"Recently, the scale of transformers has grown rapidly, which introduces considerable challenges in terms of training overhead and inference efficiency in the scope of task adaptation. Existing works, namely Parameter-Efficient Fine-Tuning (PEFT) and model compression, have separately investigated t…