Federica Merendi

On the Nature of BERT: Correlating Fine-Tuning and Linguistic Competence

Several studies in the literature on the interpretation of Neural Language Models (NLM) focus on the linguistic generalization abilities of pre-trained models. However, little attention is paid to how the linguistic knowledge of the models changes during the fine-tuning steps. In this paper, we cont…