Collision-less and Balanced Sampling for Language-Queried Audio Source Separation

Abstract

Language-queried audio source separation (LASS) is an emerging research field that has recently received increasing attention. This task aims to isolate individual sources from a mixture of signals using natural language descriptions, enabling applications in various areas such as automatic audio editing. While conventional methods focus on the system architecture, the important aspect of data processing has been overlooked. The data for training LASS are typically created by mixing various audio signals in the dataset to form a mixture. One signal is then used as the target, whereas the others are regarded as interference. However, sound events in the target signal could overlap with those in the interference signals, which may cause confusion that instructs the model to both retain and suppress the same sound events within a single training example. In addition, training LASS with large-scale datasets may suffer from the data imbalance problem, where some sound events appear too frequently while others are rare. In this paper, we address these problems by using data sampling techniques. Specifically, the interference signals are sampled so that their audio tags do not conflict with those of the target signal, where the tags are generated using an audio tagging model. To balance the data, we consider several balanced sampling approaches using tag or caption embedding. By leveraging their distribution information, we use either weighted or group sampling to boost the occurrence of underrepresented samples while reducing the presence of overrepresented ones. Experimental results show the superiority of the proposed method over state-of-the-art LASS systems in DCASE 2024 Challenge Task 9. Pre-trained model is available at: https://github.com/tucothien/LASS-CLBS.