Sound2Synth: Interpreting Sound via FM Synthesizer Parameters Estimation

Zui Chen; Yansen Jing; Shengcheng Yuan; Yifei Xu; Jian Wu; Hang Zhao

IJCAI 2022poster13 citations

Sound2Synth: Interpreting Sound via FM Synthesizer Parameters Estimation

Zui Chen, Yansen Jing, Shengcheng Yuan, Yifei Xu, Jian Wu, Hang Zhao

Abstract

Synthesizer is a type of electronic musical instrument that is now widely used in modern music production and sound design. Each parameters configuration of a synthesizer produces a unique timbre and can be viewed as a unique instrument. The problem of estimating a set of parameters configuration that best restore a sound timbre is an important yet complicated problem, i.e.: the synthesizer parameters estimation problem. We proposed a multi-modal deep-learning-based pipeline Sound2Synth, together with a network structure Prime-Dilated Convolution (PDC) specially designed to solve this problem. Our method achieved not only SOTA but also the first real-world applicable results on Dexed synthesizer, a popular FM synthesizer.

Application domains: MusicMethods and resources: Machine learning, deep learning, neural models, reinforcement learning

BibTeX

@inproceedings{ijcai2022p682,
  title     = {Sound2Synth: Interpreting Sound via FM Synthesizer Parameters Estimation},
  author    = {Chen, Zui and Jing, Yansen and Yuan, Shengcheng and Xu, Yifei and Wu, Jian and Zhao, Hang},
  booktitle = {Proceedings of the Thirty-First International Joint Conference on
               Artificial Intelligence, {IJCAI-22}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Lud De Raedt},
  pages     = {4921--4928},
  year      = {2022},
  month     = {7},
  note      = {AI and Arts},
  doi       = {10.24963/ijcai.2022/682},
  url       = {https://doi.org/10.24963/ijcai.2022/682},
}