Towards Knowledge Checking in Retrieval-augmented Generation: A Representation Perspective
Shenglai Zeng, Jiankun Zhang, Bingheng Li, Yuping Lin, Tianqi Zheng, Dante Everaert, Hanqing Lu, Hui Liu
Abstract
Retrieval-Augmented Generation (RAG) systems have shown promise in enhancing the performance of Large Language Models (LLMs). However, these systems face challenges in effectively integrating external knowledge with the LLM’s internal knowledge, often leading to issues with misleading or unhelpful information. This work aims to provide a systematic study on knowledge checking in RAG systems. We conduct a comprehensive analysis of LLM representation behaviors and demonstrate the significance of using representations in knowledge checking. Motivated by the findings, we further develop representation-based classifiers for knowledge filtering. We show substantial improvements in RAG performance, even when dealing with noisy knowledge databases. Our study provides new insights into leveraging LLM representations for enhancing the reliability and effectiveness of RAG systems.
BibTeX
@inproceedings{zeng-etal-2025-towards,
title = "Towards Knowledge Checking in Retrieval-augmented Generation: A Representation Perspective",
author = "Zeng, Shenglai and
Zhang, Jiankun and
Li, Bingheng and
Lin, Yuping and
Zheng, Tianqi and
Everaert, Dante and
Lu, Hanqing and
Liu, Hui and
Liu, Hui and
Xing, Yue and
Cheng, Monica Xiao and
Tang, Jiliang",
editor = "Chiruzzo, Luis and
Ritter, Alan and
Wang, Lu",
booktitle = "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)",
month = apr,
year = "2025",
address = "Albuquerque, New Mexico",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.naacl-long.151/",
pages = "2952--2969",
ISBN = "979-8-89176-189-6"
}