Editing Conceptual Knowledge for Large Language Models
Xiaohan Wang, Shengyu Mao, Shumin Deng, Yunzhi Yao, Yue Shen, Lei Liang, Jinjie Gu, Huajun Chen
Abstract
Recently, there has been a growing interest in knowledge editing for Large Language Models (LLMs). Current approaches and evaluations merely explore the instance-level editing, while whether LLMs possess the capability to modify concepts remains unclear. This paper pioneers the investigation of editing conceptual knowledge for LLMs, by constructing a novel benchmark dataset ConceptEdit and establishing a suite of new metrics for evaluation. The experimental results reveal that, although existing editing methods can efficiently modify concept-level definition to some extent, they also have the potential to distort the related instantial knowledge in LLMs, leading to poor performance. We anticipate this work can inspire further progress in understanding LLMs.
BibTeX
@inproceedings{wang-etal-2024-editing,
title = "Editing Conceptual Knowledge for Large Language Models",
author = "Wang, Xiaohan and
Mao, Shengyu and
Deng, Shumin and
Yao, Yunzhi and
Shen, Yue and
Liang, Lei and
Gu, Jinjie and
Chen, Huajun and
Zhang, Ningyu",
editor = "Al-Onaizan, Yaser and
Bansal, Mohit and
Chen, Yun-Nung",
booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2024",
month = nov,
year = "2024",
address = "Miami, Florida, USA",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2024.findings-emnlp.40/",
doi = "10.18653/v1/2024.findings-emnlp.40",
pages = "706--724"
}