Token-Based Audio Inpainting via Discrete Diffusion
Tali Dror, Iftach Shoham, Moshe Buchris, Oren Gal, Haim H. Permuter, Gilad Katz, Eliya Nachmani
Abstract
Audio inpainting seeks to restore missing segments in degraded recordings. Previous diffusion-based methods exhibit impaired performance when the missing region is large. We introduce the first approach that applies discrete diffusion over tokenized music representations from a pre-trained audio tokenizer, enabling stable and semantically coherent restoration of long gaps. Our method further incorporates two training approaches: a derivative-based regularization loss that enforces smooth temporal dynamics, and a span-based absorbing transition that provides structured corruption during diffusion. Experiments on the MusicNet and MAESTRO datasets with gaps up to 750ms show that our approach consistently outperforms strong baselines across range of gap lengths, for gaps of 150ms and above. This work advances musical audio restoration and introduces new directions for discrete diffusion model training. Visit our project page for examples and code.
BibTeX
@inproceedings{
dror2026tokenbased,
title={Token-based Audio Inpainting via Discrete Diffusion},
author={Tali Dror and Iftach Shoham and Moshe Buchris and Oren Gal and Haim H. Permuter and Gilad Katz and Eliya Nachmani},
booktitle={The Fourteenth International Conference on Learning Representations},
year={2026},
url={https://openreview.net/forum?id=9ZogqiyWXm}
}