An Anchor-Based Automatic Evaluation Metric for Document Summarization
Kexiang Wang, Tianyu Liu, Baobao Chang, Zhifang Sui
Abstract
The widespread adoption of reference-based automatic evaluation metrics such as ROUGE has promoted the development of document summarization. In this paper, we consider a new protocol for designing reference-based metrics that require the endorsement of source document(s). Following protocol, we propose an anchored ROUGE metric fixing each summary particle on source document, which bases the computation on more solid ground. Empirical results on benchmark datasets validate that source document helps to induce a higher correlation with human judgments for ROUGE metric. Being self-explanatory and easy-to-implement, the protocol can naturally foster various effective designs of reference-based metrics besides the anchored ROUGE introduced here.
BibTeX
@inproceedings{wang-etal-2020-anchor,
title = "An Anchor-Based Automatic Evaluation Metric for Document Summarization",
author = "Wang, Kexiang and
Liu, Tianyu and
Chang, Baobao and
Sui, Zhifang",
editor = "Scott, Donia and
Bel, Nuria and
Zong, Chengqing",
booktitle = "Proceedings of the 28th International Conference on Computational Linguistics",
month = dec,
year = "2020",
address = "Barcelona, Spain (Online)",
publisher = "International Committee on Computational Linguistics",
url = "https://aclanthology.org/2020.coling-main.500/",
doi = "10.18653/v1/2020.coling-main.500",
pages = "5696--5701"
}