Smr-Awarenet: An Adaptive Smr-Aware Neural Network for EEG Auditory Attention Guided Target Speech Extraction

Abstract

Target speech extraction remains a significant challenge in speech separation, particularly within complex multi-speaker environments where accurate isolation of the target speech and suppression of the interfering speech are crucial. Despite the progress driven by deep learning in neural network-based speech separation models, the integration of electroencephalography (EEG) signals to guide the speech separation process in complex scenes remains an open problem. In this study, we propose a novel neural network architecture called SMR-AwareNet. This architecture utilizes the Signal-to-Masker Ratio (SMR) between the target and interfering speech, along with attention information obtained through EEG auditory attention detection (AAD) module, to guide the adaptive target speech extraction. Experimental results show that the SMR-AwareNet model achieves significant improvements in Scale-Invariant Signal-to-Noise Ratio (SISNR) and consistently outperforms baseline models under various challenging conditions.