2025
LongRoPE2: Near-Lossless LLM Context Window Scaling
ICML 2025poster
LongRoPE2 is a novel approach that extends the effective context window of pre-trained large language models (LLMs) to the target length, while preserving the performance on the original shorter context window. This is achieved by three contributions: (1) a hypothesis that insufficient training in…