Shiyi Li

RESA: Bringing Back What Sparse Attention Ignores with Residual Estimation

Large Language Models (LLM) have gained significant attention. KV cache, stored to avoid quadratic complexity of attention, becomes a bottleneck due to the demands for long-context. Sparse attention (SA) has been proposed to address this by only selecting critical KVs for attention, which ma…