HANet: A Harmonic Attention-Based Network for Singing Melody Extraction from Polyphonic Music

Abstract

Singing melody extraction from polyphonic music is a complex but important task in music information retrieval. Harmonic relationships have been shown to be crucial in this task, but most existing models based on Convolutional Neural Networks (CNNs) struggle to capture long-range harmonic dependencies. To address this, we propose a Harmonic Attention-based Network (HANet) for singing melody extraction from polyphonic music, which includes multiple sampling layers. Specifically, each sampling layer uses three parallel Harmonic Attention Modules (HAMs) with CNNs of different kernel sizes to capture harmonic relationships across 1 to 6 octaves along the frequency axis. Additionally, the Channel Attention Module (CAM) is used to adaptively model long-range harmonic features across various octave ranges. Experimental results show that HANet achieves state-of-the-art performance on multiple public music melody datasets. The overall accuracy on the ADC2004 dataset reached a peak of 95.2%, with the lowest voicing false alarm result of 2.1%. Visualization results directly demonstrate that harmonic information effectively reduces octave errors. Our code is available online: https://github.com/wwsjj/HANet