Autoregressive Density Estimation Transformers for Multivariate Time Series Anomaly Detection

Abstract

Anomaly detection in multivariate time series (MTS) from sensor data is critical in many industrial applications. The challenge lies in managing massive unlabeled datasets with complex spatio-temporal correlations, diverse anomalies, and noise. While several unsupervised methods have been proposed, they are often limited to specific applications. In this paper, we introduce a probabilistic self-supervised framework, Autoregressive Density Estimation Transformer (ADET). ADET integrates an efficient transformer for learning spatio-temporal representations with density estimation networks for multi-score anomaly detection, focusing on point-to-point, point-to-distribution, and distribution-to-distribution distances. ADET improves noise resilience using optimal truncated singular value decomposition (OT-SVD) in an end-to-end optimization process. We conducted experiments by employing several encoders and performed an ablation study to examine the effect of OT-SVD.