The Touché23-ValueEval Dataset for Identifying Human Values behind Arguments
Nailia Mirzakhmedova, Johannes Kiesel, Milad Alshomary, Maximilian Heinrich, Nicolas Handke, Xiaoni Cai, Valentin Barriere, Doratossadat Dastgheib
Abstract
While human values play a crucial role in making arguments persuasive, we currently lack the necessary extensive datasets to develop methods for analyzing the values underlying these arguments on a large scale. To address this gap, we present the Touché23-ValueEval dataset, an expansion of the Webis-ArgValues-22 dataset. We collected and annotated an additional 4780 new arguments, doubling the dataset’s size to 9324 arguments. These arguments were sourced from six diverse sources, covering religious texts, community discussions, free-text arguments, newspaper editorials, and political debates. Each argument is annotated by three crowdworkers for 54 human values, following the methodology established in the original dataset. The Touché23-ValueEval dataset was utilized in the SemEval 2023 Task 4. ValueEval: Identification of Human Values behind Arguments, where an ensemble of transformer models demonstrated state-of-the-art performance. Furthermore, our experiments show that a fine-tuned large language model, Llama-2-7B, achieves comparable results.
BibTeX
@inproceedings{mirzakhmedova-etal-2024-touche23,
title = "The Touch{\'e}23-{V}alue{E}val Dataset for Identifying Human Values behind Arguments",
author = "Mirzakhmedova, Nailia and
Kiesel, Johannes and
Alshomary, Milad and
Heinrich, Maximilian and
Handke, Nicolas and
Cai, Xiaoni and
Barriere, Valentin and
Dastgheib, Doratossadat and
Ghahroodi, Omid and
SadraeiJavaheri, MohammadAli and
Asgari, Ehsaneddin and
Kawaletz, Lea and
Wachsmuth, Henning and
Stein, Benno",
editor = "Calzolari, Nicoletta and
Kan, Min-Yen and
Hoste, Veronique and
Lenci, Alessandro and
Sakti, Sakriani and
Xue, Nianwen",
booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
month = may,
year = "2024",
address = "Torino, Italia",
publisher = "ELRA and ICCL",
url = "https://aclanthology.org/2024.lrec-main.1402/",
pages = "16121--16134"
}