2025
Towards Interpretable and Efficient Attention: Compressing All by Contracting a Few
NeurIPS 2025spotlight
Attention mechanisms have achieved significant empirical success in multiple fields, but their underlying optimization objectives remain unclear yet. Moreover, the quadratic complexity of self-attention has become increasingly prohibitive. Although interpretability and efficiency are two mutually re…