Segment-Recurrent Transformer with Multi-Scale Fusion for Long-Term Time Series Forecasting

Abstract

Long-term time series forecasting (LTSF) seeks to make accurate long-term predictions by leveraging extensive historical data, which is crucial for solving scientific and engineering challenges. Traditional transformer-based methods process historical segments individually, leading to a limited view that overlooks distant dependencies within the entire time series. In this paper, we introduce the Segment-Recurrent Transformer (SRTrans), designed to provide a more comprehensive understanding of historical time series dynamics. By incorporating segment-level recurrence into the Transformer, our model enhances inter-segment information flow, capturing longer-term and global dependencies. We also propose a multi-scale adaptive fusion module that efficiently integrates diverse patterns using a variable-scale chunking mechanism and a weight-mixing strategy. Additionally, our spectrum purge operation improves data preprocessing by extracting significant long-term patterns from the frequency domain. Extensive experiments on eight real-world datasets demonstrate SRTrans’s effectiveness in accuracy and efficiency, offering a promising new solution for LTSF tasks.