Neal Lawton

Neural Architecture Search for Parameter-Efficient Fine-tuning of Large Pre-trained Language Models

Parameter-efficient tuning (PET) methods fit pre-trained language models (PLMs) to downstream tasks by either computing a small compressed update for a subset of model parameters, or appending and fine-tuning a small number of new model parameters to the pre-trained network. Hand-designed PET archit…