Position Paper: MeMo: Towards Language Models with Associative Memory Mechanisms
Fabio Massimo Zanzotto, Elena Sofia Ruzzetti, Giancarlo A. Xompero, Leonardo Ranaldi, Davide Venditti, Federico Ranaldi, Cristina Giannone, Andrea Favalli
Abstract
Memorization is a fundamental ability of Transformer-based Large Language Models, achieved through learning. In this position/theory paper, we propose a paradigm shift by designing an architecture to memorize text directly, bearing in mind the principle that memorization precedes learning. We introduce MeMo, a novel architecture for language modeling that explicitly memorizes sequences of tokens in layered associative memories. By design, MeMo offers transparency and the possibility of model editing, including forgetting texts. We experimented with the MeMo architecture, showing the memorization power of the one-layer and the multi-layer configurations.
BibTeX
@inproceedings{zanzotto-etal-2025-position,
title = "Position Paper: {M}e{M}o: Towards Language Models with Associative Memory Mechanisms",
author = "Zanzotto, Fabio Massimo and
Ruzzetti, Elena Sofia and
Xompero, Giancarlo A. and
Ranaldi, Leonardo and
Venditti, Davide and
Ranaldi, Federico and
Giannone, Cristina and
Favalli, Andrea and
Romagnoli, Raniero",
editor = "Che, Wanxiang and
Nabende, Joyce and
Shutova, Ekaterina and
Pilehvar, Mohammad Taher",
booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
month = jul,
year = "2025",
address = "Vienna, Austria",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.findings-acl.785/",
doi = "10.18653/v1/2025.findings-acl.785",
pages = "15169--15180",
ISBN = "979-8-89176-256-5"
}