Yitian Huang

Pretraining Context Compressor for Large Language Models with Embedding-Based Memory

Efficient processing of long contexts in large language models (LLMs) is essential for real-world applications like retrieval-augmented generation and in-context learning, especially in resource-constrained environments such as edge computing. This paper explores the embedding-based context compress…