IJCAI 2024poster25 citations

MusicMagus: Zero-Shot Text-to-Music Editing via Diffusion Models

Yixiao Zhang, Yukara Ikemiya, Gus Xia, Naoki Murata, Marco A. Martínez-Ramírez, Wei-Hsiang Liao, Yuki Mitsufuji, Simon Dixon

Abstract

Recent advances in text-to-music generation models have opened new avenues in musical creativity. However, the task of editing these generated music remains a significant challenge. This paper introduces a novel approach to edit music generated by such models, enabling the modification of specific attributes, such as genre, mood, and instrument, while maintaining other aspects unchanged. Our method transforms text editing to the latent space manipulation, and adds an additional constraint to enforce consistency. It seamlessly integrates with existing pretrained text-to-music diffusion models without requiring additional training. Experimental results demonstrate superior performance over both zero-shot and certain supervised baselines in style and timbre transfer evaluations. We also show the practical applicability of our approach in real-world music editing scenarios.

Application domains: Music and soundMethods and resources: Machine learning, deep learning, neural models, reinforcement learning
BibTeX
@inproceedings{ijcai2024p864,
  title     = {MusicMagus: Zero-Shot Text-to-Music Editing via Diffusion Models},
  author    = {Zhang, Yixiao and Ikemiya, Yukara and Xia, Gus and Murata, Naoki and Martínez-Ramírez, Marco A. and Liao, Wei-Hsiang and Mitsufuji, Yuki and Dixon, Simon},
  booktitle = {Proceedings of the Thirty-Third International Joint Conference on
               Artificial Intelligence, {IJCAI-24}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Kate Larson},
  pages     = {7805--7813},
  year      = {2024},
  month     = {8},
  note      = {AI, Arts & Creativity},
  doi       = {10.24963/ijcai.2024/864},
  url       = {https://doi.org/10.24963/ijcai.2024/864},
}
MusicMagus: Zero-Shot Text-to-Music Editing via Diffusion Models · IJCAI 2024