Distractor Generation based on Text2Text Language Models with Pseudo Kullback-Leibler Divergence Regulation
Hui-Juan Wang, Kai-Yu Hsieh, Han-Cheng Yu, Jui-Ching Tsou, Yu An Shih, Chen-Hua Huang, Yao-Chung Fan
Abstract
In this paper, we address the task of cloze-style multiple choice question (MCQs) distractor generation. Our study is featured by the following designs. First, we propose to formulate the cloze distractor generation as a Text2Text task. Second, we propose pseudo Kullback-Leibler Divergence for regulating the generation to consider the item discrimination index in education evaluation. Third, we explore the candidate augmentation strategy and multi-tasking training with cloze-related tasks to further boost the generation performance. Through experiments with benchmarking datasets, our best perfomring model advances the state-of-the-art result from 10.81 to 22.00 (p@1 score).
BibTeX
@inproceedings{wang-etal-2023-distractor,
title = "Distractor Generation based on {T}ext2{T}ext Language Models with Pseudo {K}ullback-{L}eibler Divergence Regulation",
author = "Wang, Hui-Juan and
Hsieh, Kai-Yu and
Yu, Han-Cheng and
Tsou, Jui-Ching and
Shih, Yu An and
Huang, Chen-Hua and
Fan, Yao-Chung",
editor = "Rogers, Anna and
Boyd-Graber, Jordan and
Okazaki, Naoaki",
booktitle = "Findings of the Association for Computational Linguistics: ACL 2023",
month = jul,
year = "2023",
address = "Toronto, Canada",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2023.findings-acl.790/",
doi = "10.18653/v1/2023.findings-acl.790",
pages = "12477--12491"
}