HYMAN: Hybrid Memory and Attention Network for Unsupervised Anomaly Detection

Abstract

Detecting anomalies in unsupervised multivariate time series is challenging due to the intricate temporal patterns present in both local short-term and global long-term dependencies. Long short-term memory has achieved impressive results in this domain, yet it is gradually being supplemented by Transformers, due to limitations such as non-parallelization, gradient vanishing, and difficulty in focusing on local information. Leveraging the attention mechanism, Transformers can attend to all time steps simultaneously, effectively capturing local dependencies. However, they may often face challenges in efficiently modeling long-term dependencies, particularly in real-world scenarios. To address these issues, we propose the HYbrid Memory and Attention Network (HYMAN), which integrates memory and attention mechanisms together to model both global and local information. The attention captures short-term dependencies by focusing on temporal autocorrelation, while the memory stores and updates key historical patterns in global information, facilitating the learning of long-term dependencies. In contrast to previous approaches, HYMAN eliminates the need for auxiliary loss, simplifying the training by reducing the effort for coefficients tuning. In the inference phase, HYMAN introduces a novel anomaly scoring method that fuses features from both the temporal and latent spaces, offering high-performance detection compared to traditional methods that rely solely on reconstruction. Extensive experiments on real-world benchmark datasets demonstrate that HYMAN achieves state-of-the-art performance by leveraging the complementary strengths of memory and attention mechanisms.