Down the Toxicity Rabbit Hole: A  Framework to Bias Audit Large Language Models with Key Emphasis on Racism, Antisemitism, and Misogyny

Arka Dutta; Adel Khorramrouz; Sujan Dutta; Ashiqur R. KhudaBukhsh

IJCAI 2024poster8 citations

Down the Toxicity Rabbit Hole: A Framework to Bias Audit Large Language Models with Key Emphasis on Racism, Antisemitism, and Misogyny

Arka Dutta, Adel Khorramrouz, Sujan Dutta, Ashiqur R. KhudaBukhsh

Abstract

This paper makes three contributions. First, it presents a generalizable, novel framework dubbed toxicity rabbit hole that iteratively elicits toxic content from a wide suite of large language models. Spanning a set of 1,266 identity groups, we first conduct a bias audit of PaLM 2 guardrails presenting key insights. Next, we report generalizability across several other models. Through the elicited toxic content, we present a broad analysis with a key emphasis on racism, antisemitism, misogyny, Islamophobia, homophobia, and transphobia. We release a massive dataset of machine-generated toxic content with a view toward safety for all. Finally, driven by concrete examples, we discuss potential ramifications.

AI Ethics, Trust, Fairness: GeneralNatural Language Processing: General

BibTeX

@inproceedings{ijcai2024p801,
  title     = {Down the Toxicity Rabbit Hole: A  Framework to Bias Audit Large Language Models with Key Emphasis on Racism, Antisemitism, and Misogyny},
  author    = {Dutta, Arka and Khorramrouz, Adel and Dutta, Sujan and KhudaBukhsh, Ashiqur R.},
  booktitle = {Proceedings of the Thirty-Third International Joint Conference on
               Artificial Intelligence, {IJCAI-24}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Kate Larson},
  pages     = {7242--7250},
  year      = {2024},
  month     = {8},
  note      = {AI for Good},
  doi       = {10.24963/ijcai.2024/801},
  url       = {https://doi.org/10.24963/ijcai.2024/801},
}