JiakaiWang JiakaiWang

E2-LLM: Efficient and Extreme Length Extension of Large Language Models

Training Large Language Models (LLMs) to process extensive context lengths incurs prohibitive computational costs. Prevailing techniques for extending context capabilities in LLMs typically require not only additional training procedures but also access to datasets with long context (e.g., sequences…