Qingyan Zhao

Representation Degeneration Problem in Prompt-based Models for Natural Language Understanding

Prompt-based fine-tuning (PF), by aligning with the training objective of pre-trained language models (PLMs), has shown improved performance on many few-shot natural language understanding (NLU) benchmarks. However, the word embedding space of PLMs exhibits anisotropy, which is called the representa…