EpMAN: Episodic Memory AttentioN for Generalizing to Longer Contexts
Subhajit Chaudhury, Payel Das, Sarathkrishna Swaminathan, Georgios Kollias, Elliot Nelson, Khushbu Pahwa, Tejaswini Pedapati, Igor Melnyk
Abstract
Recent advances in Large Language Models (LLMs) have yielded impressive successes on many language tasks. However, efficient processing of long contexts using LLMs remains a significant challenge. We introduce **EpMAN** – a method for processing long contexts in an episodic memory module while holistically attending to semantically-relevant context chunks. Output from episodic attention is then used to reweigh the decoder’s self-attention to the stored KV cache of the context during training and generation. When an LLM decoder is trained using **EpMAN**, its performance on multiple challenging single-hop long-context recall and question-answering benchmarks is found to be stronger and more robust across the range from 16k to 256k tokens than baseline decoders trained with self-attention, and popular retrieval-augmented generation frameworks.
BibTeX
@inproceedings{chaudhury-etal-2025-epman,
title = "{E}p{MAN}: Episodic Memory {A}ttentio{N} for Generalizing to Longer Contexts",
author = "Chaudhury, Subhajit and
Das, Payel and
Swaminathan, Sarathkrishna and
Kollias, Georgios and
Nelson, Elliot and
Pahwa, Khushbu and
Pedapati, Tejaswini and
Melnyk, Igor and
Riemer, Matthew",
editor = "Che, Wanxiang and
Nabende, Joyce and
Shutova, Ekaterina and
Pilehvar, Mohammad Taher",
booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
month = jul,
year = "2025",
address = "Vienna, Austria",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.acl-long.574/",
doi = "10.18653/v1/2025.acl-long.574",
pages = "11696--11708",
ISBN = "979-8-89176-251-0"
}