PriorSinger: Singing Voice Synthesis Model with Prior Condition Cross Attention

Abstract

The singing voice synthesis system is designed to generate realistic and expressive singing based on a given musical score. Generative Adversarial Networks (GANs) or diffusion models generate acoustic features, such as Mel-spectrograms, which are subsequently reconstructed into waveforms by a vocoder. In this work, the musical score is encoded as a prior condition to guide the diffusion denoiser through a novel prior cross-attention Transformer during the denoising process. Moreover, we introduce attention mechanisms in both the time and frequency domains within the diffusion denoiser to enhance the resolution of the generated acoustic features. Additionally, incorporating rotary positional encoding allows the model to better handle temporal and frequency positional information. Our model is capable of synthesizing singing with both higher quality and more vivid expressiveness. In subjective evaluations of the Opencpop dataset, our model outperforms state-of-the-art methods.