Limitations of Human Identification of Automatically Generated Text
Nadège Alavoine, Maximin Coavoux, Emmanuelle Esperança-Rodier, Romane Gallienne, Carlos-Emiliano González-Gallardo, Jérôme Goulian, Jose G. Moreno, Aurélie Névéol
Abstract
Neural text generation is receiving broad attention with the publication of new tools such as ChatGPT. The main reason for that is that the achieved quality of the generated text may be attributed to a human writer by the naked eye of a human evaluator. In this paper, we propose a new corpus in French and English for the task of recognising automatically generated texts and we conduct a study of how humans perceive the text. Our results show, as previous work before the ChatGPT era, that the generated texts by tools such as ChatGPT share some common characteristics but they are not clearly identifiable which generates different perceptions of these texts.
BibTeX
@inproceedings{alavoine-etal-2024-limitations,
title = "Limitations of Human Identification of Automatically Generated Text",
author = "Alavoine, Nad{\`e}ge and
Coavoux, Maximin and
Esperan{\c{c}}a-Rodier, Emmanuelle and
Gallienne, Romane and
Gonz{\'a}lez-Gallardo, Carlos-Emiliano and
Goulian, J{\'e}r{\^o}me and
Moreno, Jose G. and
N{\'e}v{\'e}ol, Aur{\'e}lie and
Schwab, Didier and
Segonne, Vincent and
Simoens, Johanna",
editor = "Calzolari, Nicoletta and
Kan, Min-Yen and
Hoste, Veronique and
Lenci, Alessandro and
Sakti, Sakriani and
Xue, Nianwen",
booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
month = may,
year = "2024",
address = "Torino, Italia",
publisher = "ELRA and ICCL",
url = "https://aclanthology.org/2024.lrec-main.919/",
pages = "10511--10516"
}