InjectTST: Injecting Global Information into Independent Channels for Long Time Series Forecasting

Abstract

Transformer has become one of the most popular architectures for multivariate time series (MTS) forecasting. However, existing Transformer-based methods still lack consideration of cross-time-and-channel dependency modeling, which is important to MTS forecasting. In addition, existing methods either completely ignore the channel dependency for robustness or model channel dependency at the sacrifice of robustness. How to achieve a balance between robustness and information capacity has not been investigated. To address these problems, a Transformer-based method, InjectTST, is proposed in this paper. Instead of designing a channel-dependent model directly, we retain the channel-independent backbone and restrainedly inject global information into individual channels. A joint temporal-channel attention scheme is proposed to better model the global information and a token dropout mechanism is designed to improve the model robustness and calculation efficiency. Through a Transformer-based injection module, the independent channels get improved by concentrating on useful global information. Experiments indicate that InjectTST can achieve stable improvement compared with state-of-the-art methods.