Towards Reference-free Text Simplification Evaluation with a BERT Siamese Network Architecture
Xinran Zhao, Esin Durmus, Dit-Yan Yeung
Abstract
Text simplification (TS) aims to modify sentences to make their both content and structure easier to understand. Traditional n-gram matching-based TS evaluation metrics heavily rely on the exact token match and human-annotated simplified sentences. In this paper, we present a novel neural-network-based reference-free TS metric BETS that leverages pre-trained contextualized language representation models and large-scale paraphrasing datasets to evaluate simplicity and meaning preservation. We show that our metric, without collecting any costly human simplification reference, correlates better than existing metrics with human judgments for the quality of both overall simplification (+7.7%) and its key aspects, i.e., comparative simplicity (+11.2%) and meaning preservation (+9.2%).
BibTeX
@inproceedings{zhao-etal-2023-towards,
title = "Towards Reference-free Text Simplification Evaluation with a {BERT} {S}iamese Network Architecture",
author = "Zhao, Xinran and
Durmus, Esin and
Yeung, Dit-Yan",
editor = "Rogers, Anna and
Boyd-Graber, Jordan and
Okazaki, Naoaki",
booktitle = "Findings of the Association for Computational Linguistics: ACL 2023",
month = jul,
year = "2023",
address = "Toronto, Canada",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2023.findings-acl.838/",
doi = "10.18653/v1/2023.findings-acl.838",
pages = "13250--13264"
}