Xinyan Wen

Automatic Design of Adapter Architectures for Enhanced Parameter-Efficient Fine-Tuning

Adapters, one of the most important parameter-efficient fine-tuning (PEFT) methods, achieve state-of-the-art (SOTA) performance through manual architecture design. To unlock the full potential of adapter tuning, we introduce the AutoAdapter framework, designated to design novel adapter architectures…