2025
Rodimus*: Breaking the Accuracy-Efficiency Trade-Off with Efficient Attentions
ICLR 2025poster
Recent advancements in Transformer-based large language models (LLMs) have set new standards in natural language processing. However, the classical softmax attention incurs significant computational costs, leading to a $O(T)$ complexity for per-token generation, where $T$ represents the context leng…