TSIformer: Multi-Scale Dilation Transformer with Cross-variable and Cross-feature Dependency for Time Series Imputation

Abstract

In the time series imputation task, most Transformer-based methods adopt the standard full attention mechanism, which not only has high complexity but also cannot aggregate semantic multi-scale information effectively. This paper proposes a Transformer-based model called TSIformer for time series imputation. TSIformer proposes a Sequence Multi-Scale Dilated Attention mechanism that assigns distinct dilation rates to different attention heads to enable the ability of multi-scale representation learning for series data. Self-attention is performed over temporal segments, through a dilated sliding window to capture dependencies within the sequence. TSIformer further introduces a Dual-Path Convolutional Feed-Forward Network to replace the traditional Feed-Forward Network in vanilla Transformer to better capture both the cross-feature and cross-variable dependency. Through extensive experiments on multiple benchmarks, TSIformer demonstrates its effectiveness and robustness.