Semi-Supervised Speech Enhancement Based On Speech Purity

Abstract

We tend to assume most available speech corpora we use are either completely clean or completely noised. However, the reality is most of them are a mix of both. In this paper, we propose a semi-supervised speech enhancement framework to enhance such typical speech datasets. This framework includes an estimator to measure the speech purity. Utterances with high speech purity are considered clean, otherwise noised. For clean speech utterances, we follow the supervised learning mechanism to train a deep learning speech enhancement model. For noised speech, we update the model in an unsupervised manner. Hence, we design our training loss as a combination of the supervised loss and unsupervised loss. We refer to this framework as SemiEnhance. Experimental results show that SemiEnhance substantially improves the speech quality, and achieves new state-of-the-art results on benchmark datasets: 2022 DNS Challenge and NoiseX-92.