Multivariate Time Series Data Mining for Failure Prediction & Root Cause Analysis

Abstract

This paper presents an analysis of a large-scale, high-dimensional industrial dataset containing over 2 million data points collected over several months. The dataset includes more than 200 failures of various types, each resulting from complex causes. Utilizing state-of-the-art unsupervised multivariate anomaly detection algorithms, a system was developed that predicts failures several minutes in advance, introducing a new evaluation metric termed lead time. This system also identifies the location and potential causes of these failures. Initial application of current anomaly detection algorithms yielded low F1 scores, due to either low recall or low precision. To address this issue, a visual reasoning framework was created to reduce false positives by analyzing motifs and discords in the top-N anomalous signals contributing to the anomaly. We find that the human-in-the-loop approach enhances the precision and F1-score of multivariate algorithms by leveraging human judgment for final decision-making. Another key finding is that the LSTM-based anomaly detection algorithm achieves sufficient lead time for the industrial failure prediction studied in this paper.