2025
DAM: Dynamic Attention Mask for Long-Context Large Language Model Inference Acceleration
ACL 2025finding
Long-context understanding is crucial for many NLP applications, yet transformers struggle with efficiency due to the quadratic complexity of self-attention. Sparse attention methods alleviate this cost but often impose static, predefined masks, failing to capture heterogeneous attention patterns. T…