Caimming Xiong

Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting

Existing studies addressing gender bias of pre-trained language models, usually build a small gender-neutral data set and conduct a second phase pre-training on the model with such data. However, given the limited size and concentrated focus of the gender-neutral data, catastrophic forgetting would…