AIConfPaper
← Search

Anna Sotnikova

5 accepted papers

2025

INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge

Angelika Romanou, Negar Foroutan, Anna Sotnikova, Sree Harsha Nelaturu, Shivalika Singh, Rishabh Maheshwary +51
ICLR 2025spotlight

The performance differential of large language models (LLM) between languages hinders their effective deployment in many regions, inhibiting the potential economic and societal value of generative AI tools in many communities. However, the development of functional LLMs in many languages (i.e., mult…

Cited by 9Source ↗PDF ↗Scholar ↗
2025

Measuring what Matters: Construct Validity in Large Language Model Benchmarks

Andrew M. Bean, Ryan Othniel Kearns, Angelika Romanou, Franziska Sofia Hafner, Harry Mayne, Jan Batzner +36
NeurIPS 2025poster

Evaluating large language models (LLMs) is crucial for both assessing their capabilities and identifying safety or robustness issues prior to deployment. Reliably measuring abstract and complex phenomena such as `safety' and `robustness' requires strong construct validity, that is, having measures t…

Cited by 0Source ↗Scholar ↗
2024

“Flex Tape Can’t Fix That”: Bias and Misinformation in Edited Language Models

Karina H Halevy, Anna Sotnikova, Badr AlKhamissi, Syrielle Montariol, Antoine Bosselut
EMNLP 2024main

Weight-based model editing methods update the parametric knowledge of language models post-training. However, these methods can unintentionally alter unrelated parametric knowledge representations, potentially increasing the risk of harm. In this work, we investigate how weight editing methods unexp…

Cited by 4Source ↗PDF ↗code ↗Scholar ↗
2023

Which Examples Should be Multiply Annotated? Active Learning When Annotators May Disagree

Connor Baumler, Anna Sotnikova, Hal Daumé III
ACL 2023findings

Linguistic annotations, especially for controversial topics like hate speech detection, are frequently contested due to annotator backgrounds and positionalities. In such situations, preserving this disagreement through the machine learning pipeline can be important for downstream use cases. However…

Cited by 26Source ↗PDF ↗code ↗Scholar ↗
2022

Theory-Grounded Measurement of U.S. Social Stereotypes in English Language Models

Yang Trista Cao, Anna Sotnikova, Hal Daumé III, Rachel Rudinger, Linda Zou
NAACL 2022long

NLP models trained on text have been shown to reproduce human stereotypes, which can magnify harms to marginalized groups when systems are deployed at scale. We adapt the Agency-Belief-Communion (ABC) stereotype model of Koch et al. (2016) from social psychology as a framework for the systematic stu…

Cited by 57Source ↗PDF ↗code ↗Scholar ↗