MST-HA: Multi-Modal Signal Fusion with Bayesian Optimization for Robust Industrial Robot Joint Health Assessment

Abstract

This paper presents a novel multi-modal deep learning framework for industrial robot joint health assessment and prediction, leveraging non-invasive signal fusion and Bayesian optimization. The proposed method addresses the challenges of comprehensive joint state monitoring in complex industrial environments without disrupting normal operations. We integrate Hall-effect current sensors, external accelerometers, and joint encoders to collect multi-modal data, including motor currents, vibrations, and kinematic information. A novel Adaptive Multi-Receptive Field Attention Network (AMRFAN) is employed to extract features from each modality, while a synchrosqueezing transform (SST) is utilized to capture time-frequency characteristics. An attention mechanism dynamically adjusts the weights of different modalities, and a bidirectional long short-term memory (BiLSTM) network models the temporal dependencies in the fused features. To enhance model performance and generalization, we implement a Bayesian optimization framework for hyperparameter tuning. Furthermore, we incorporate a Bayesian neural network to quantify prediction uncertainties, providing reliability metrics for decision-making processes. Experimental results on a four-axis industrial robot demonstrate that our framework achieves a 98.3% accuracy in joint health state classification and a mean absolute error of 4.2 in remaining useful life prediction, outperforming state-of-the-art single-modality methods. The proposed approach offers a robust, adaptable solution for real-time health monitoring and predictive maintenance of industrial robot joints, potentially improving manufacturing efficiency and reliability.