Developing a Reliable, Fast, General-Purpose Hallucination Detection and Mitigation Service
Song Wang, Xun Wang, Jie Mei, Yujia Xie, Si-Qing Chen, Wayne Xiong
Abstract
Hallucination, a phenomenon where large language models (LLMs) produce output that is factually incorrect or unrelated to the input, is a major challenge for LLM applications that require accuracy and dependability. In this paper, we introduce a reliable and high-speed production system aimed at detecting and rectifying the hallucination issue within LLMs. Our system encompasses named entity recognition (NER), natural language inference (NLI), span-based detection (SBD), and an intricate decision tree-based process to reliably detect a wide range of hallucinations in LLM responses. Furthermore, we have crafted a rewriting mechanism that maintains an optimal mix of precision, response time, and cost-effectiveness. We detail the core elements of our framework and underscore the paramount challenges tied to response time, availability, and performance metrics, which are crucial for real-world deployment of these technologies. Our extensive evaluation, utilizing offline data and live production traffic, confirms the efficacy of our proposed framework and service.
BibTeX
@inproceedings{wang-etal-2025-developing-reliable,
title = "Developing a Reliable, Fast, General-Purpose Hallucination Detection and Mitigation Service",
author = "Wang, Song and
Wang, Xun and
Mei, Jie and
Xie, Yujia and
Chen, Si-Qing and
Xiong, Wayne",
editor = "Chen, Weizhu and
Yang, Yi and
Kachuee, Mohammad and
Fu, Xue-Yong",
booktitle = "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 3: Industry Track)",
month = apr,
year = "2025",
address = "Albuquerque, New Mexico",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.naacl-industry.72/",
pages = "971--978",
ISBN = "979-8-89176-194-0"
}