2024
Can Mamba Learn How To Learn? A Comparative Study on In-Context Learning Tasks
ICML 2024poster
State-space models (SSMs), such as Mamba (Gu & Dao, 2023), have been proposed as alternatives to Transformer networks in language modeling, incorporating gating, convolutions, and input-dependent token selection to mitigate the quadratic cost of multi-head attention. Although SSMs exhibit competitiv…