2026
Local Linear Attention: An Optimal Interpolation of Linear and Softmax Attention For Test-Time Regression
ICLR 2026poster
Transformer architectures have achieved remarkable success in various domains. While efficient alternatives to Softmax Attention have been widely studied, the search for more expressive mechanisms grounded in theoretical insight—even at greater computational cost—has been relatively underexplored. I…