Fangxu Liu

2 accepted papers

2026

Prism: Spectral-Aware Block-Sparse Attention

Xinghao Wang, Pengyu Wang, Xiaoran Liu, Fangxu Liu, Jason Chu, Kai Song +1

ICML 2026poster

Block-sparse attention is promising for accelerating long-context LLM pre-filling, yet identifying relevant blocks efficiently remains a bottleneck. Existing methods typically employ coarse-grained attention as a proxy for block importance estimation, but often resort to expensive token-level search…

Cited by 0 Source ↗Scholar ↗

2026

Sparser Block-Sparse Attention via Token Permutation

Xinghao Wang, Pengyu Wang, Dong Zhang, Chenkun Tan, Shaojun Zhou, Zhaoxiang Liu +4

ICML 2026poster

Scaling the context length of large language models (LLMs) offers significant benefits but is computationally expensive. This expense stems primarily from the self-attention mechanism, whose $O(N^2)$ complexity with respect to sequence length presents a major bottleneck for both memory and latency. …

Cited by 0 Source ↗Scholar ↗