Direct Preference Optimization for Speech Autoregressive Diffusion Models

Zhijun Liu; Dongya Jia; Shuai Wang; Zhuo Chen; Haizhou Li

ICASSP 2026poster0 citations

Direct Preference Optimization for Speech Autoregressive Diffusion Models

Zhijun Liu, Dongya Jia, Shuai Wang, Zhuo Chen, Haizhou Li

Abstract

Autoregressive diffusion models (ARDMs) have recently been applied to speech generation, achieving state-of-the-art (SOTA) performance in zero-shot text-to-speech. By autoregressively generating continuous speech tokens with next-token diffusion, these models offer a promising alternative to next-token prediction, avoiding the technical complexities associated with discrete speech tokenization. As a relatively new paradigm, research on reinforcement learning (RL)-based fine-tuning of speech ARDMs remains limited. In this paper, we propose Autoregressive Diffusion-Direct Preference Optimization (ARDM-DPO) to advance this research. By fine-tuning the recently proposed zero-shot text-to-speech model DiTAR with DPO, we achieve significant improvements in terms of speech expressiveness and robustness for long texts.

BibTeX

@inproceedings{icassp2026_directpreference,
  title = {Direct Preference Optimization for Speech Autoregressive Diffusion Models},
  author = {Zhijun Liu and Dongya Jia and Shuai Wang and Zhuo Chen and Haizhou Li},
  booktitle = {ICASSP 2026},
  year = {2026}
}