Asformer: Learning From Adjacent Scale

Abstract

Long-term series forecasting is crucial in real-world applications. Existing works have leveraged self-attention mechanisms and multi-scale temporal data to learn complex dependencies and patterns in long-term series prediction. However, most prediction models directly aggregate together multi-scale data resulting in the inability to appropriately capture key implicit information at each time step and lack of flexibility. To alleviate this problem, in this paper, we propose a novel Transformer-based network, dubbed ASformer, which can capture the spatio-temporal dependence of time series from adjacent scale features while obtaining global information. Specifically, we design a Scale Bonder that empowers ASformer with progressive capacities in capturing the correlations between data of two adjacent scales. We achieve the state-of-the-art performances with a 20.18% relative improvement on three benchmarks covering two practical applications: energy and disease. Our code will be made publicly available at: https://github.com/Hanpengjiang/ASformer.