2023
On Robustness of Finetuned Transformer-based NLP Models
Pavan Kalyan Reddy Neerudu, SUBBA REDDY OOTA, mounika marreddy, venkateswara Rao Kagita, Manish Gupta
EMNLP 2023long findings
Transformer-based pretrained models like BERT, GPT-2 and T5 have been finetuned for a large number of natural language processing (NLP) tasks, and have been shown to be very effective. However, while finetuning, what changes across layers in these models with respect to pretrained checkpoints is und…