SiMBA-TS: Simplified Channel Mixing and Mamba for Long-term Time Series Forecasting

Abstract

We investigate the question of whether transformers are effective for long-term time series forecasting. They suffer from temporal information loss, even though they use positional encoding to embed order information. This is due to the inherent permutation-invariant nature of attention nets. Mamba has a stability issue at scale, so we use a novel EinFFT method for getting positive Eigenvalues with the help of fast Fourier transforms. We introduce SiMBA-TS, a novel approach combining the selective scan SSM called Mamba for token mixing and a new method called EinFFT for channel mixing. Our benchmark studies on seven-time series datasets demonstrate that SiMBA-TS achieves state-of-the-art performance, outperforming transformers such as PatchTST, TimeNet, and CrossFormer. Additionally, our evaluation of a proprietary industrial dataset confirms that SiMBA-TS is the leading model for time series forecasting.