← Search

William Isaac

3 accepted papers

2025

Century: A Framework and Dataset for Evaluating Historical Contextualisation of Sensitive Images

ICLR 2025spotlight

How do multi-modal generative models describe images of recent historical events and figures, whose legacies may be nuanced, multifaceted, or contested? This task necessitates not only accurate visual recognition, but also socio-cultural knowledge and cross-modal reasoning. To address this evaluati…

Cited by 0SourcePDFScholar
2024

STAR: SocioTechnical Approach to Red Teaming Language Models

EMNLP 2024main

This research introduces STAR, a sociotechnical framework that improves on current best practices for red teaming safety of large language models. STAR makes two key contributions: it enhances steerability by generating parameterised instructions for human red teamers, leading to improved coverage o…

Cited by 13SourcePDFScholar
2022

Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models

NeurIPS 2022accept

Large language models produce human-like text that drive a growing number of applications. However, recent literature and, increasingly, real world observations, have demonstrated that these models can generate language that is toxic, biased, untruthful or otherwise harmful. Though work to evalua…

Cited by 55SourcePDFScholar