2024
Neurocache: Efficient Vector Retrieval for Long-range Language Modeling
NAACL 2024long
This paper introduces Neurocache, an approach to extend the effective context size of large language models (LLMs) using an external vector cache to store its past states. Like recent vector retrieval approaches, Neurocache uses an efficient k-nearest-neighbor (kNN) algorithm to retrieve relevant pa…