TCTformer: Long-term forecasting with dual attention transformers

Abstract

In the field of multi-variable long-term time series (MLTS) prediction, many deep learning models have been developed, and Transformer-based models have received widespread attention for their ability to capture the complex interactions between sequences. However, as the sequence length increases and the inter-channel connections become more tightly coupled, the existing Transformer-based models exhibit limited order-preserving and coupling feature capabilities. To address this issue, we introduce TCTformer, a dual attention Transformer model that aims to combine and optimize the model’s ability to model the longitudinal dynamic evolution along the time dimension and consider the cross-channel coupling effect. We validated that the temporal and channel attention mechanisms in TCTformer complement each other, enabling modeling of deep-level features in MLTS, thereby effectively enhancing the model’s order-preserving and coupled feature capabilities. A series of extensive experiments on seven real-world datasets demonstrated the effectiveness of TCTformer.