Improving Autoregressive Grammatical Error Correction with Non-autoregressive Models
Hang Cao, Zhiquan Cao, Chi Hu, Baoyu Hou, Tong Xiao, Jingbo Zhu
Abstract
Grammatical Error Correction (GEC) aims to correct grammatical errors in sentences. We find that autoregressive models tend to assign low probabilities to tokens that need corrections. Here we introduce additional signals to the training of GEC models so that these systems can learn to better predict at ambiguous positions. To do this, we use a non-autoregressive model as an auxiliary model, and develop a new regularization term of training by considering the difference in predictions between the autoregressive and non-autoregressive models. We experiment with this method on both English and Chinese GEC tasks. Experimental results show that our GEC system outperforms the baselines on all the data sets significantly.
BibTeX
@inproceedings{cao-etal-2023-improving,
title = "Improving Autoregressive Grammatical Error Correction with Non-autoregressive Models",
author = "Cao, Hang and
Cao, Zhiquan and
Hu, Chi and
Hou, Baoyu and
Xiao, Tong and
Zhu, Jingbo",
editor = "Rogers, Anna and
Boyd-Graber, Jordan and
Okazaki, Naoaki",
booktitle = "Findings of the Association for Computational Linguistics: ACL 2023",
month = jul,
year = "2023",
address = "Toronto, Canada",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2023.findings-acl.760/",
doi = "10.18653/v1/2023.findings-acl.760",
pages = "12014--12027"
}