TFPSNet: Time-Frequency Domain Path Scanning Network for Speech Separation

Abstract

Speech separation has been very successful with deep learning techniques. In this paper, we propose time-frequency (T-F) domain path scanning network (TFPSNet) for speech separation task. The connections between frequency bins in frequency path, time path, and T-F path are modeled by transformer. We also introduce T-F path loss function to improve the performance further. The proposed TFPSNet could learn more details of frequency structure and separate the feature in T-F domain. Experiments show that proposed model achieves state-of-the-art (SOTA) performance on public WSJ0-2mix datasets. It reaches 21.1dB SI-SDRi on WSJ0-2mix, and 19.7dB SI-SDRi on Libri-2mix. Furthermore, our approach has good generalizability. The model trained on WSJ0-2mix dataset achieves 18.6dB SI-SDRi on Libri-2mix test set without any fine-tuning work. This result is even 0.4dB higher than DPTNet trained on Libri-2mix dataset.