A Multi-Modal Information Fusion Model for Automatic Sleep Staging

Abstract

Sleep staging is a key step in understanding sleep mechanisms and their impact on the human body. We usually use multi-modal sleep signals to enhance the sensitivity of the sleep staging model, in which the shared information between modals and the specific information of each modal play the key role in identifying different sleep stages. Existing studies directly extract information from multi-modal sleep signals without distinguishing between shared information and specific information, which may contain redundant information as they reused shared information from different modals. Moreover, not all modal-specific information is equally valuable for sleep staging, as some of it might be mere noise. To cope with these problems, we introduce a novel multi-modal information fusion model for automatic sleep staging. Our model uses a multi-stream structure to extract cross-modal shared and modal-specific information, respectively, and uses the information fusion module to integrate modal-specific information with shared information sequentially based on their contributions to sleep staging. Experimental evaluations confirm that our model outperforms the comparison models, and incorporating both the multi-modal shared-specific information separation strategy and the information fusion module into the sleep staging framework enhances its identification ability.