COLING 2025main0 citations
Addressing the Training-Inference Discrepancy in Discrete Diffusion for Text Generation
Abstract
This study addresses the discrepancy between training and inference in discrete diffusion models for text generation. We propose two novel strategies: (1) a training schema that considers two-step diffusion processes, allowing the model to use its own predicted output as input for subsequent steps during training and (2) a scheduling technique that gradually increases the probability of using self-generated text as training progresses. Experiments conducted on four widely used text generation benchmark datasets demonstrate that both proposed strategies improve the performance of discrete diffusion models in text generation.
BibTeX
@inproceedings{asada-miwa-2025-addressing,
title = "Addressing the Training-Inference Discrepancy in Discrete Diffusion for Text Generation",
author = "Asada, Masaki and
Miwa, Makoto",
editor = "Rambow, Owen and
Wanner, Leo and
Apidianaki, Marianna and
Al-Khalifa, Hend and
Eugenio, Barbara Di and
Schockaert, Steven",
booktitle = "Proceedings of the 31st International Conference on Computational Linguistics",
month = jan,
year = "2025",
address = "Abu Dhabi, UAE",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.coling-main.477/",
pages = "7156--7164"
}