Speech Separation Based on Pre-trained Model and Deep Modularization

Abstract

This work proposes a speech separation technique that clusters spectrogram points or raw speech blocks by combining traditional graph clustering objectives with deep neural networks. We first extract features from spectrogram points or speech blocks using a pre-trained model, then apply deep modularization to cluster these features. This approach identifies clusters dominated by each speaker in mixed speech. Extensive evaluations demonstrate that our technique is competitive with fully supervised state-of-the-art speech separation methods.