← Search

Nicole Meister

5 accepted papers

2025

Benchmarking Distributional Alignment of Large Language Models

NAACL 2025long

Language models (LMs) are increasingly used as simulacra for people, yet their ability to match the distribution of views of a specific demographic group and be distributionally aligned remains uncertain. This notion of distributional alignment is complex, as there is significant variation in the ty…

2024

Proving Test Set Contamination in Black-Box Language Models

ICLR 2024oral

Large language models are trained on vast amounts of internet data, prompting concerns that they have memorized public benchmarks. Detecting this type of contamination is challenging because the pretraining data used by proprietary models are often not publicly accessible. We propose a procedure fo…

2023

Gender Artifacts in Visual Datasets

ICCV 2023poster

Gender biases are known to exist within large-scale visual datasets and can be reflected or even amplified in downstream models. Many prior works have proposed methods for mitigating gender biases, often by attempting to remove gender expression information from images. To understand the feasibility…

Cited by 36PDFScholar
2022

HIVE: Evaluating the Human Interpretability of Visual Explanations

ECCV 2022poster

"As AI technology is increasingly applied to high-impact, high-risk domains, there have been a number of new methods aimed at making AI models more human interpretable. Despite the recent growth of interpretability work, there is a lack of systematic evaluation of proposed techniques. In this work,…

2022

MACRONYM: A Large-Scale Dataset for Multilingual and Multi-Domain Acronym Extraction

COLING 2022main

Acronym extraction is the task of identifying acronyms and their expanded forms in texts that is necessary for various NLP applications. Despite major progress for this task in recent years, one limitation of existing AE research is that they are limited to the English language and certain domains (…

Cited by 22SourcePDFScholar