AIConfPaper
← Search

Fatemehsadat Mireshghallah

12 accepted papers

2026

Position: Don't Just "Fix it in Post'': A Science of AI Must Study Learning Dynamics

Stella Biderman, Mohammad Aflah Khan, Fatemehsadat Mireshghallah, Catherine Arnett, Fazl Barez, Naomi Saphra
ICML 2026oral

What would it mean to have a *scientific* understanding of AI? Language models are not static objects—they are snapshots of time-evolving processes shaped by data, objectives, and optimization dynamics. Yet the field predominantly treats models as fixed artifacts, analyzing behaviors after training …

Cited by 0Source ↗Scholar ↗
2026

Privasis: Synthesizing the Largest "Public" Private Dataset from Scratch

Hyunwoo Kim, Fatemehsadat Mireshghallah, Michael Duan, Rui Xin, Stella Li, Jaehun Jung +8
ICML 2026poster

Research involving privacy-sensitive data has always been constrained by data scarcity, standing in sharp contrast to other areas that have benefited from data scaling. To quench this thirst, we present Privasis (i.e., privacy oasis), the first million-scale fully synthetic dataset entirely built fr…

Cited by 0Source ↗Scholar ↗
2024

Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generation

Xinyu Tang, Richard Shin, Huseyin A Inan, Andre Manoel, Fatemehsadat Mireshghallah, Zinan Lin +3
ICLR 2024poster

We study the problem of in-context learning (ICL) with large language models (LLMs) on private datasets. This scenario poses privacy risks, as LLMs may leak or regurgitate the private examples demonstrated in the prompt. We propose a novel algorithm that generates synthetic few-shot demonstrations…

Cited by 66Source ↗PDF ↗code ↗Scholar ↗
2023

Membership Inference Attacks against Language Models via Neighbourhood Comparison

Justus Mattern, Fatemehsadat Mireshghallah, Zhijing Jin, Bernhard Schoelkopf, Mrinmaya Sachan, Taylor Berg-Kirkpatrick
ACL 2023findings

Membership Inference attacks (MIAs) aim to predict whether a data sample was present in the training data of a machine learning model or not, and are widely used for assessing the privacy risks of language models. Most existing attacks rely on the observation that models tend toassign higher probabi…

Cited by 180Source ↗PDF ↗code ↗Scholar ↗
2023

Privacy-Preserving Domain Adaptation of Semantic Parsers

Fatemehsadat Mireshghallah, Yu Su, Tatsunori Hashimoto, Jason Eisner, Richard Shin
ACL 2023long

Task-oriented dialogue systems often assist users with personal or confidential matters. For this reason, the developers of such a system are generally prohibited from observing actual usage. So how can they know where the system is failing and needs more training data or new functionality? In this…

Cited by 14Source ↗PDF ↗code ↗Scholar ↗
2022

An Empirical Analysis of Memorization in Fine-tuned Autoregressive Language Models

Fatemehsadat Mireshghallah, Archit Uniyal, Tianhao Wang, David Evans, Taylor Berg-Kirkpatrick
EMNLP 2022main

Large language models are shown to present privacy risks through memorization of training data, andseveral recent works have studied such risks for the pre-training phase. Little attention, however, has been given to the fine-tuning phase and it is not well understood how different fine-tuning metho…

Cited by 88Source ↗PDF ↗Scholar ↗
2022

Differentially Private Model Compression

Fatemehsadat Mireshghallah, Arturs Backurs, Huseyin A Inan, Lukas Wutschitz, Janardhan Kulkarni
NeurIPS 2022accept

Recent papers have shown that large pre-trained language models (LLMs) such as BERT, GPT-2 can be fine-tuned on private data to achieve performance comparable to non-private models for many downstream Natural Language Processing (NLP) tasks while simultaneously guaranteeing differential privacy. The…

Cited by 24Source ↗PDF ↗Scholar ↗
2022

Mix and Match: Learning-free Controllable Text Generationusing Energy Language Models

Fatemehsadat Mireshghallah, Kartik Goyal, Taylor Berg-Kirkpatrick
ACL 2022long

Recent work on controlled text generation has either required attribute-based fine-tuning of the base language model (LM), or has restricted the parameterization of the attribute discriminator to be compatible with the base autoregressive LM. In this work, we propose Mix and Match LM, a global score…

Cited by 81Source ↗PDF ↗code ↗Scholar ↗
2022

Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks

Fatemehsadat Mireshghallah, Kartik Goyal, Archit Uniyal, Taylor Berg-Kirkpatrick, Reza Shokri
EMNLP 2022main

The wide adoption and application of Masked language models (MLMs) on sensitive data (from legal to medical) necessitates a thorough quantitative investigation into their privacy vulnerabilities. Prior attempts at measuring leakage of MLMs via membership inference attacks have been inconclusive, imp…

Cited by 177Source ↗PDF ↗Scholar ↗
2022

UserIdentifier: Implicit User Representations for Simple and Effective Personalized Sentiment Analysis

Fatemehsadat Mireshghallah, Vaishnavi Shrivastava, Milad Shokouhi, Taylor Berg-Kirkpatrick, Robert Sim, Dimitrios Dimitriadis
NAACL 2022long

Global models are typically trained to be as generalizable as possible. Invariance to the specific user is considered desirable since models are shared across multitudes of users. However, these models are often unable to produce personalized responses for individual users, based on their data. Cont…

Cited by 39Source ↗PDF ↗Scholar ↗
2021

Privacy Regularization: Joint Privacy-Utility Optimization in LanguageModels

Fatemehsadat Mireshghallah, Huseyin Inan, Marcello Hasegawa, Victor Rühle, Taylor Berg-Kirkpatrick, Robert Sim
NAACL 2021long

Neural language models are known to have a high capacity for memorization of training samples. This may have serious privacy im- plications when training models on user content such as email correspondence. Differential privacy (DP), a popular choice to train models with privacy guarantees, comes wi…

Cited by 44Source ↗PDF ↗Scholar ↗
2021

Style Pooling: Automatic Text Style Obfuscation for Improved Classification Fairness

Fatemehsadat Mireshghallah, Taylor Berg-Kirkpatrick
EMNLP 2021main

Text style can reveal sensitive attributes of the author (e.g. age and race) to the reader, which can, in turn, lead to privacy violations and bias in both human and algorithmic decisions based on text. For example, the style of writing in job applications might reveal protected attributes of the ca…

Cited by 15Source ↗PDF ↗code ↗Scholar ↗