Ngoc Bich Uyen Vo

MuonSSM: Orthogonalizing State Space Models for Sequence Modeling

State-space models (SSMs) have emerged as efficient linear-time alternatives to attention for long-sequence modeling. However, existing SSMs often suffer from instability and memory degradation over extended horizons due to poorly conditioned first-order updates and uncontrolled spectral geometry. W…