Fact Checking Machine Generated Text with Dependency Trees
Alex Estes, Nikhita Vedula, Marcus Collins, Matt Cecil, Oleg Rokhlenko
Abstract
Factual and logical errors made by Natural Language Generation (NLG) systems limit their applicability in many settings. We study this problem in a conversational search and recommendation setting, and observe that we can often make two simplifying assumptions in this domain: (i) there exists a body of structured knowledge we can use for verifying factuality of generated text; and (ii) the text to be factually assessed typically has a well-defined structure and style. Grounded in these assumptions, we propose a fast, unsupervised and explainable technique, DepChecker, that assesses factuality of input text based on rules derived from structured knowledge patterns and dependency relations with respect to the input text. We show that DepChecker outperforms state-of-the-art, general purpose fact-checking techniques in this special, but important case.
BibTeX
@inproceedings{estes-etal-2022-fact,
title = "Fact Checking Machine Generated Text with Dependency Trees",
author = "Estes, Alex and
Vedula, Nikhita and
Collins, Marcus and
Cecil, Matt and
Rokhlenko, Oleg",
editor = "Li, Yunyao and
Lazaridou, Angeliki",
booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: Industry Track",
month = dec,
year = "2022",
address = "Abu Dhabi, UAE",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.emnlp-industry.46/",
doi = "10.18653/v1/2022.emnlp-industry.46",
pages = "458--466"
}