ACL 2022findings12 citations
Constructing Open Cloze Tests Using Generation and Discrimination Capabilities of Transformers
Mariano Felice, Shiva Taslimipoor, Paula Buttery
Abstract
This paper presents the first multi-objective transformer model for generating open cloze tests that exploits generation and discrimination capabilities to improve performance. Our model is further enhanced by tweaking its loss function and applying a post-processing re-ranking algorithm that improves overall test structure. Experiments using automatic and human evaluation show that our approach can achieve up to 82% accuracy according to experts, outperforming previous work and baselines. We also release a collection of high-quality open cloze tests along with sample system output and human annotations that can serve as a future benchmark.
BibTeX
@inproceedings{felice-etal-2022-constructing,
title = "Constructing Open Cloze Tests Using Generation and Discrimination Capabilities of Transformers",
author = "Felice, Mariano and
Taslimipoor, Shiva and
Buttery, Paula",
editor = "Muresan, Smaranda and
Nakov, Preslav and
Villavicencio, Aline",
booktitle = "Findings of the Association for Computational Linguistics: ACL 2022",
month = may,
year = "2022",
address = "Dublin, Ireland",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.findings-acl.100/",
doi = "10.18653/v1/2022.findings-acl.100",
pages = "1263--1273"
}