ICLR 2023top-25%533 citations
Mass-Editing Memory in a Transformer
Kevin Meng, Arnab Sen Sharma, Alex J Andonian, Yonatan Belinkov, David Bau
Abstract
Recent work has shown exciting promise in updating large language models with new memories, so as to replace obsolete information or add specialized knowledge. However, this line of work is predominantly limited to updating single associations. We develop MEMIT, a method for directly updating a language model with many memories, demonstrating experimentally that it can scale up to thousands of associations for GPT-J (6B) and GPT-NeoX (20B), exceeding prior work by an order of magnitude. Our code and data will be open-sourced upon publication.
language modelsGPTtransformersmodel editingfactual associationsmemory
BibTeX
@inproceedings{
meng2023massediting,
title={Mass-Editing Memory in a Transformer},
author={Kevin Meng and Arnab Sen Sharma and Alex J Andonian and Yonatan Belinkov and David Bau},
booktitle={The Eleventh International Conference on Learning Representations },
year={2023},
url={https://openreview.net/forum?id=MkbcAHIYgyS}
}