2025
Fixed-Point RNNs: Interpolating from Diagonal to Dense
NeurIPS 2025spotlight
Linear recurrent neural networks (RNNs) and state-space models (SSMs) such as Mamba have become promising alternatives to softmax-attention as sequence mixing layers in Transformer architectures. Current models, however, do not exhibit the full state-tracking expressivity of RNNs because they rely o…