LLM generated responses to mitigate the impact of hate speech
Jakub Podolak, Szymon Łukasik, Paweł Balawender, Jan Ossowski, Jan Piotrowski, Katarzyna Bakowicz, Piotr Sankowski
Abstract
In this study, we explore the use of Large Language Models (LLMs) to counteract hate speech. We conducted the first real-life A/B test assessing the effectiveness of LLM-generated counter-speech. During the experiment, we posted 753 automatically generated responses aimed at reducing user engagement under tweets that contained hate speech toward Ukrainian refugees in Poland.Our work shows that interventions with LLM-generated responses significantly decrease user engagement, particularly for original tweets with at least ten views, reducing it by over 20%. This paper outlines the design of our automatic moderation system, proposes a simple metric for measuring user engagement and details the methodology of conducting such an experiment. We discuss the ethical considerations and challenges in deploying generative AI for discourse moderation.
BibTeX
@inproceedings{podolak-etal-2024-llm,
title = "{LLM} generated responses to mitigate the impact of hate speech",
author = "Podolak, Jakub and
{\L}ukasik, Szymon and
Balawender, Pawe{\l} and
Ossowski, Jan and
Piotrowski, Jan and
Bakowicz, Katarzyna and
Sankowski, Piotr",
editor = "Al-Onaizan, Yaser and
Bansal, Mohit and
Chen, Yun-Nung",
booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2024",
month = nov,
year = "2024",
address = "Miami, Florida, USA",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2024.findings-emnlp.931/",
doi = "10.18653/v1/2024.findings-emnlp.931",
pages = "15860--15876"
}