Dialogue Language Model with Large-Scale Persona Data Engineering
Mengze Hong, Chen Jason Zhang, Chaotao Chen, Rongzhong Lian, Di Jiang
Abstract
Maintaining persona consistency is paramount in the application of open-domain dialogue systems, as exemplified by models like ChatGPT. Despite significant advancements, the limited scale and diversity of current persona dialogue datasets remain challenges to achieving robust persona-consistent dialogue models. In this study, drawing inspiration from the success of large-scale pre-training, we introduce PPDS, an open-domain persona dialogue system that employs extensive generative pre-training on a persona dialogue dataset to enhance persona consistency. Specifically, we present a persona extraction model designed to autonomously and precisely generate vast persona dialogue datasets. Additionally, we unveil a pioneering persona augmentation technique to address the invalid persona bias inherent in the constructed dataset. Both quantitative and human evaluations consistently highlight the superior response quality and persona consistency of our proposed model, underscoring its effectiveness.
BibTeX
@inproceedings{hong-etal-2025-dialogue,
title = "Dialogue Language Model with Large-Scale Persona Data Engineering",
author = "Hong, Mengze and
Zhang, Chen Jason and
Chen, Chaotao and
Lian, Rongzhong and
Jiang, Di",
editor = "Chen, Weizhu and
Yang, Yi and
Kachuee, Mohammad and
Fu, Xue-Yong",
booktitle = "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 3: Industry Track)",
month = apr,
year = "2025",
address = "Albuquerque, New Mexico",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.naacl-industry.71/",
pages = "961--970",
ISBN = "979-8-89176-194-0"
}