Global Enhanced Frame Prompt Tuning for Sound Event Detection

Abstract

Sound Event Detection (SED) often employs pre-trained models to address data scarcity issues. However, existing systems usually treat the pretrained models as frozen feature extractors, resulting in suboptimal efficiency, or fully fine-tune the pretrained models, which requires substantial computational resources. To fully leverage the knowledge from pretrained models, we propose a novel Global Enhanced Frame Prompt Tuning (GE-FPT) framework, providing global and local insights tailored for SED tasks. Additionally, Frame Prompt Tuning (FPT) is proposed in our GE-FPT to effectively explore local temporal information, i.e., temporal details and context, which is essential for SED tasks, and in particular, for precise event boundary detection. Extensive experiments claim that our approach significantly outperforms full fine-tuning methods while substantially reducing computational costs. Our system achieves new state-of-the-art results, with PSDS1/PSDS2 scores of 0.628/0.845 on the DCASE2023 Challenge Task4 dataset. The source code is publicly available<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>.