Investigating person-specific errors in chat-oriented dialogue systems
Koh Mitsuda, Ryuichiro Higashinaka, Tingxuan Li, Sen Yoshida
Abstract
Creating chatbots to behave like real people is important in terms of believability. Errors in general chatbots and chatbots that follow a rough persona have been studied, but those in chatbots that behave like real people have not been thoroughly investigated. We collected a large amount of user interactions of a generation-based chatbot trained from large-scale dialogue data of a specific character, i.e., target person, and analyzed errors related to that person. We found that person-specific errors can be divided into two types: errors in attributes and those in relations, each of which can be divided into two levels: self and other. The correspondence with an existing taxonomy of errors was also investigated, and person-specific errors that should be addressed in the future were clarified.
BibTeX
@inproceedings{mitsuda-etal-2022-investigating,
title = "Investigating person-specific errors in chat-oriented dialogue systems",
author = "Mitsuda, Koh and
Higashinaka, Ryuichiro and
Li, Tingxuan and
Yoshida, Sen",
editor = "Muresan, Smaranda and
Nakov, Preslav and
Villavicencio, Aline",
booktitle = "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)",
month = may,
year = "2022",
address = "Dublin, Ireland",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.acl-short.50/",
doi = "10.18653/v1/2022.acl-short.50",
pages = "464--469"
}