EMNLP 2022finding17 citations
Sequentially Controlled Text Generation
Alexander Spangher, Yao Ming, Xinyu Hua, Nanyun Peng
Abstract
While GPT-2 generates sentences that are remarkably human-like, longer documents can ramble and do not follow human-like writing structure. We study the problem of imposing structure on long-range text. We propose a novel controlled text generation task, sequentially controlled text generation, and identify a dataset, NewsDiscourse as a starting point for this task. We develop a sequential controlled text generation pipeline with generation and editing. We test different degrees of structural awareness and show that, in general, more structural awareness results in higher control- accuracy, grammaticality, coherency and topicality, approaching human-level writing performance.
BibTeX
@inproceedings{spangher-etal-2022-sequentially,
title = "Sequentially Controlled Text Generation",
author = "Spangher, Alexander and
Ming, Yao and
Hua, Xinyu and
Peng, Nanyun",
editor = "Goldberg, Yoav and
Kozareva, Zornitsa and
Zhang, Yue",
booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2022",
month = dec,
year = "2022",
address = "Abu Dhabi, United Arab Emirates",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.findings-emnlp.509/",
doi = "10.18653/v1/2022.findings-emnlp.509",
pages = "6848--6866"
}