2025
Do Transformer Interpretability Methods Transfer to RNNs?
AAAI 2025technical
Recent advances in recurrent neural network architectures, such as Mamba and RWKV, have enabled RNNs to match or exceed the performance of equal-size transformers in terms of language modeling perplexity and downstream evaluations, suggesting that future systems may be built on completely new archit…