EMNLP 2022main13 citations
Linguistic Corpus Annotation for Automatic Text Simplification Evaluation
Rémi Cardon, Adrien Bibal, Rodrigo Wilkens, David Alfter, Magali Norré, Adeline Müller, Watrin Patrick, Thomas François
Abstract
Evaluating automatic text simplification (ATS) systems is a difficult task that is either performed by automatic metrics or user-based evaluations. However, from a linguistic point-of-view, it is not always clear on what bases these evaluations operate. In this paper, we propose annotations of the ASSET corpus that can be used to shed more light on ATS evaluation. In addition to contributing with this resource, we show how it can be used to analyze SARI’s behavior and to re-evaluate existing ATS systems. We present our insights as a step to improve ATS evaluation protocols in the future.
BibTeX
@inproceedings{cardon-etal-2022-linguistic,
title = "Linguistic Corpus Annotation for Automatic Text Simplification Evaluation",
author = {Cardon, R{\'e}mi and
Bibal, Adrien and
Wilkens, Rodrigo and
Alfter, David and
Norr{\'e}, Magali and
M{\"u}ller, Adeline and
Patrick, Watrin and
Fran{\c{c}}ois, Thomas},
editor = "Goldberg, Yoav and
Kozareva, Zornitsa and
Zhang, Yue",
booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing",
month = dec,
year = "2022",
address = "Abu Dhabi, United Arab Emirates",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.emnlp-main.121/",
doi = "10.18653/v1/2022.emnlp-main.121",
pages = "1842--1866"
}