2026
Long-Context Modeling with Dynamic Hierarchical Sparse Attention for Memory-Constrained LLM Inference
ICML 2026spotlight
The quadratic cost of attention limits the scalability of long-context LLMs, especially under limited hardware memory budgets. While attention is often sparse, existing static sparse methods cannot adapt to task- or input-dependent variations, and recent dynamic approaches rely on predefined templat…