Minji Jung

Is Prompt Transfer Always Effective? An Empirical Study of Prompt Transfer for Question Answering

Prompt tuning, which freezes all parameters of a pre-trained model and only trains a soft prompt, has emerged as a parameter-efficient approach. For the reason that the prompt initialization becomes sensitive when the model size is small, the prompt transfer that uses the trained prompt as an initia…