AIConfPaper
← Search

Kamalika Chaudhuri

57 accepted papers

2026

CIMemories: A Compositional Benchmark For Contextual Integrity In LLMs

Niloofar Mireshghallah, Neal Mangaokar, Narine Kokhlikyan, Arman Zharmagambetov, Manzil Zaheer, Saeed Mahloujifar +1
ICLR 2026poster

Large Language Models (LLMs) increasingly use persistent memory from past interactions to enhance personalization and task performance. However, this memory creates critical risks when sensitive information is revealed in inappropriate contexts. We present CIMemories, a benchmark for evaluating whet…

Cited by 0Source ↗code ↗Scholar ↗
2026

How much can language models memorize?

John Morris, Chawin Sitawarin, Narine Kokhlikyan, Chuan Guo, Edward Suh, Alexander Rush +2
ICML 2026oral

We propose a new method for estimating how much a model knows about a datapoint and use it to measure the capacity of modern language models. Prior studies of language model memorization have struggled to disentangle memorization from generalization. We formally separate memorization into two compon…

Cited by 0Source ↗Scholar ↗
2026

Safety Alignment of LMs via Non-cooperative Games

Anselm Paulus, Ilia Kulikov, Brandon Amos, REMI MUNOS, Ivan Evtimov, Kamalika Chaudhuri +1
ICML 2026spotlight

Ensuring the safety of language models (LMs) while maintaining their usefulness remains a critical challenge in AI alignment. Current approaches rely on sequential adversarial training: generating adversarial prompts and fine-tuning LMs to defend against them. We introduce a different paradigm: fram…

Cited by 0Source ↗Scholar ↗
2025

AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions

Polina Kirichenko, Mark Ibrahim, Kamalika Chaudhuri, Samuel Bell
NeurIPS 2025poster

For Large Language Models (LLMs) to be reliably deployed in both everyday and high-stakes domains, knowing when not to answer is equally critical as answering correctly. Real-world user queries, which can be underspecified, ill-posed, or fundamentally unanswerable, require LLMs to reason about uncer…

Cited by 0Source ↗code ↗Scholar ↗
2025

AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents

Arman Zharmagambetov, Chuan Guo, Ivan Evtimov, Maya Pavlova, Ruslan Salakhutdinov, Kamalika Chaudhuri
NeurIPS 2025poster

Autonomous AI agents that can follow instructions and perform complex multi-step tasks have tremendous potential to boost human productivity. However, to perform many of these tasks, the agents need access to personal information from their users, raising the question of whether they are capable of…

Cited by 0Source ↗code ↗Scholar ↗
2025

Can We Infer Confidential Properties of Training Data from LLMs?

Pengrun Huang, Chhavi Yadav, Kamalika Chaudhuri, Ruihan Wu
NeurIPS 2025spotlight

Large language models (LLMs) are increasingly fine-tuned on domain-specific datasets to support applications in fields such as healthcare, finance, and law. These fine-tuning datasets often have sensitive and confidential dataset-level properties — such as patient demographics or disease prevalence—…

Cited by 0Source ↗Scholar ↗
2025

Do LLMs Really Forget? Evaluating Unlearning with Knowledge Correlation and Confidence Awareness

Rongzhe Wei, Peizhi Niu, Hans Hao-Hsun Hsu, Ruihan Wu, Haoteng Yin, Mohsen Ghassemi +6
NeurIPS 2025poster

Machine unlearning techniques aim to mitigate unintended memorization in large language models (LLMs). However, existing approaches predominantly focus on the explicit removal of isolated facts, often overlooking latent inferential dependencies and the non-deterministic nature of knowledge within LL…

Cited by 0Source ↗Scholar ↗
2025

ExpProof : Operationalizing Explanations for Confidential Models with ZKPs

Chhavi Yadav, Evan Laufer, Dan Boneh, Kamalika Chaudhuri
ICML 2025poster

In principle, explanations are intended as a way to increase trust in machine learning models and are often obligated by regulations. However, many circumstances where these are demanded are adversarial in nature, meaning the involved parties have misaligned interests and are incentivized to manipul…

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2025

Rethinking the Role of Verbatim Memorization in LLM Privacy

Tom Sander, Bargav Jayaraman, Mark Ibrahim, Kamalika Chaudhuri, Chuan Guo
NeurIPS 2025poster

Conventional wisdom in machine learning privacy research states that memorization directly implies a loss of privacy. In contrast, a well-generalized model only remembers distributional patterns and preserves privacy of its training data. In this work, we show that this relationship is much more c…

Cited by 0Source ↗Scholar ↗
2025

WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks

Ivan Evtimov, Arman Zharmagambetov, Aaron Grattafiori, Chuan Guo, Kamalika Chaudhuri
NeurIPS 2025poster

Autonomous UI agents powered by AI have tremendous potential to boost human productivity by automating routine tasks such as filing taxes and paying bills. However, a major challenge in unlocking their full potential is security, which is exacerbated by the agent's ability to take action on their us…

Cited by 0Source ↗code ↗Scholar ↗
2024

Differentially Private Representation Learning via Image Captioning

Tom Sander, Yaodong Yu, Maziar Sanjabi, Alain Oliviero Durmus, Yi Ma, Kamalika Chaudhuri +1
ICML 2024poster

Differentially private (DP) machine learning is considered the gold-standard solution for training a model from sensitive data while still preserving privacy. However, a major barrier to achieving this ideal is its sub-optimal privacy-accuracy trade-off, which is particularly visible in DP represent…

Cited by 7Source ↗PDF ↗code ↗Scholar ↗
2024

Effective pruning of web-scale datasets based on complexity of concept clusters

Amro Kamal Mohamed Abbas, Evgenia Rusak, Kushal Tirumala, Wieland Brendel, Kamalika Chaudhuri, Ari S. Morcos
ICLR 2024poster

Utilizing massive web-scale datasets has led to unprecedented performance gains in machine learning models, but also imposes outlandish compute requirements for their training. In order to improve training and data efficiency, we here push the limits of pruning large-scale multimodal datasets for tr…

Cited by 26Source ↗PDF ↗code ↗Scholar ↗
2024

FairProof : Confidential and Certifiable Fairness for Neural Networks

Chhavi Yadav, Amrita Roy Chowdhury, Dan Boneh, Kamalika Chaudhuri
ICML 2024poster

Machine learning models are increasingly used in societal applications, yet legal and privacy concerns demand that they very often be kept confidential. Consequently, there is a growing distrust about the fairness properties of these models in the minds of consumers, who are often at the receiving e…

Cited by 8Source ↗PDF ↗code ↗Scholar ↗
2024

Measuring Dejavu Memorization Efficiently

Narine Kokhlikyan, Bargav Jayaraman, Florian Bordes, Chuan Guo, Kamalika Chaudhuri
NeurIPS 2024poster

Recent research has shown that representation learning models may accidentally memorize their training data. For example, the déjà vu method shows that for certain representation learning models and training images, it is sometimes possible to correctly predict the foreground label given only the re…

Cited by 1Source ↗PDF ↗code ↗Scholar ↗
2024

ViP: A Differentially Private Foundation Model for Computer Vision

Yaodong Yu, Maziar Sanjabi, Yi Ma, Kamalika Chaudhuri, Chuan Guo
ICML 2024oral

Artificial intelligence (AI) has seen a tremendous surge in capabilities thanks to the use of foundation models trained on internet-scale data. On the flip side, the uncurated nature of internet-scale data also poses significant privacy and legal risks, as they often contain personal information or…

Cited by 18Source ↗PDF ↗code ↗Scholar ↗
2023

Do SSL Models Have Déjà Vu? A Case of Unintended Memorization in Self-supervised Learning

Casey Meehan, Florian Bordes, Pascal Vincent, Kamalika Chaudhuri, Chuan Guo
NeurIPS 2023poster

Self-supervised learning (SSL) algorithms can produce useful image representations by learning to associate different parts of natural images with one another. However, when taken to the extreme, SSL models can unintendedly memorize specific parts in individual training samples rather than learning…

Cited by 13Source ↗PDF ↗code ↗Scholar ↗
2023

Privacy-Aware Compression for Federated Learning Through Numerical Mechanism Design

Chuan Guo, Kamalika Chaudhuri, Pierre Stock, Michael Rabbat
ICML 2023poster

In private federated learning (FL), a server aggregates differentially private updates from a large number of clients in order to train a machine learning model. The main challenge in this setting is balancing privacy with both classification accuracy of the learnt model as well as the number of bit…

Cited by 9Source ↗PDF ↗Scholar ↗
2023

Why does Throwing Away Data Improve Worst-Group Error?

Kamalika Chaudhuri, Kartik Ahuja, Martin Arjovsky, David Lopez-Paz
ICML 2023oral

When facing data with imbalanced classes or groups, practitioners follow an intriguing strategy to achieve best results. They throw away examples until the classes or groups are balanced in size, and then perform empirical risk minimization on the reduced training set. This opposes common wisdom in…

Cited by 18Source ↗PDF ↗Scholar ↗
2022

Bounding Training Data Reconstruction in Private (Deep) Learning

Chuan Guo, Brian Karrer, Kamalika Chaudhuri, Laurens van der Maaten
ICML 2022oral

Differential privacy is widely accepted as the de facto method for preventing data leakage in ML, and conventional wisdom suggests that it offers strong protection against privacy attacks. However, existing semantic guarantees for DP focus on membership inference, which may overestimate the adversar…

Cited by 72Source ↗PDF ↗code ↗Scholar ↗
2022

Thompson Sampling for Robust Transfer in Multi-Task Bandits

Zhi Wang, Chicheng Zhang, Kamalika Chaudhuri
ICML 2022spotlight

We study the problem of online multi-task learning where the tasks are performed within similar but not necessarily identical multi-armed bandit environments. In particular, we study how a learner can improve its overall performance across multiple related tasks through robust transfer of knowledge.…

Cited by 9Source ↗PDF ↗code ↗Scholar ↗
2021

Approximate Data Deletion from Machine Learning Models

Zachary Izzo, Mary Anne Smart, Kamalika Chaudhuri, James Zou
AISTATS 2021poster

Deleting data from a trained machine learning (ML) model is a critical task in many applications. For example, we may want to remove the influence of training points that might be out of date or outliers. Regulations such as EU’s General Data Protection Regulation also stipulate that individuals can…

Cited by 328Source ↗PDF ↗code ↗Scholar ↗
2021

Connecting Interpretability and Robustness in Decision Trees through Separation

Michal Moshkovitz, Yao-Yuan Yang, Kamalika Chaudhuri
ICML 2021spotlight

Recent research has recognized interpretability and robustness as essential properties of trustworthy classification. Curiously, a connection between robustness and interpretability was empirically observed, but the theoretical reasoning behind it remained elusive. In this paper, we rigorously inves…

Cited by 30Source ↗PDF ↗code ↗Scholar ↗
2021

Multitask Bandit Learning Through Heterogeneous Feedback Aggregation

Zhi Wang, Chicheng Zhang, Manish Kumar Singh, Laurel Riek, Kamalika Chaudhuri
AISTATS 2021poster

In many real-world applications, multiple agents seek to learn how to perform highly related yet slightly different tasks in an online bandit learning protocol. We formulate this problem as the $\epsilon$-multi-player multi-armed bandit problem, in which a set of players concurrently interact with a…

Cited by 30Source ↗PDF ↗code ↗Scholar ↗
2021

Revisiting Model-Agnostic Private Learning: Faster Rates and Active Learning

Chong Liu, Yuqing Zhu, Kamalika Chaudhuri, Yu-Xiang Wang
AISTATS 2021poster

The Private Aggregation of Teacher Ensembles (PATE) framework is one of the most promising recent approaches in differentially private learning. Existing theoretical analysis shows that PATE consistently learns any VC-classes in the realizable setting, but falls short in explaining its success in mo…

Cited by 17Source ↗PDF ↗Scholar ↗
2021

Sample Complexity of Robust Linear Classification on Separated Data

Robi Bhattacharjee, Somesh Jha, Kamalika Chaudhuri
ICML 2021spotlight

We consider the sample complexity of learning with adversarial robustness. Most prior theoretical results for this problem have considered a setting where different classes in the data are close together or overlapping. We consider, in contrast, the well-separated case where there exists a classifie…

Cited by 27Source ↗PDF ↗Scholar ↗
2020

A Closer Look at Accuracy vs. Robustness

Yao-Yuan Yang, Cyrus Rashtchian, Hongyang Zhang, Ruslan Salakhutdinov, Kamalika Chaudhuri
NeurIPS 2020poster

Current methods for training robust networks lead to a drop in test accuracy, which has led prior works to posit that a robustness-accuracy tradeoff may be inevitable in deep learning. We take a closer look at this phenomenon and first show that real image datasets are actually separated. With this…

Cited by 385Source ↗PDF ↗code ↗Scholar ↗
2020

A Three Sample Hypothesis Test for Evaluating Generative Models

Casey Meehan, Kamalika Chaudhuri, Sanjoy Dasgupta
AISTATS 2020poster

Detecting overfitting in generative models is an important challenge in machine learning. In this work, we formalize a form of overfitting that we call {\em{data-copying}} – where the generative model memorizes and outputs training samples or small variations thereof. We provide a three sample test…

Cited by 6Source ↗PDF ↗Scholar ↗
2020

Robustness for Non-Parametric Classification: A Generic Attack and Defense

Yao-Yuan Yang, Cyrus Rashtchian, Yizhen Wang, Kamalika Chaudhuri
AISTATS 2020poster

Adversarially robust machine learning has received much recent attention. However, prior attacks and defenses for non-parametric classifiers have been developed in an ad-hoc or classifier-specific basis. In this work, we take a holistic look at adversarial examples for non-parametric classifiers, in…

Cited by 53Source ↗PDF ↗code ↗Scholar ↗
2020

Variational Bayes in Private Settings (VIPS) (Extended Abstract)

James R. Foulds, Mijung Park, Kamalika Chaudhuri, Max Welling
IJCAI 2020poster

Many applications of Bayesian data analysis involve sensitive information such as personal documents or medical records, motivating methods which ensure that privacy is protected. We introduce a general privacy-preserving framework for Variational Bayes (VB), a widely used optimization-based Bayesia…

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2019

The Label Complexity of Active Learning from Observational Data

Songbai Yan, Kamalika Chaudhuri, Tara Javidi
NeurIPS 2019poster

Counterfactual learning from observational data involves learning a classifier on an entire population based on data that is observed conditioned on a selection policy. This work considers this problem in an active setting, where the learner additionally has access to unlabeled examples and can choo…

Cited by 11Source ↗PDF ↗code ↗Scholar ↗
2018

Analyzing the Robustness of Nearest Neighbors to Adversarial Examples

Yizhen Wang, Somesh Jha, Kamalika Chaudhuri
ICML 2018oral

Motivated by safety-critical applications, test-time attacks on classifiers via adversarial examples has recently received a great deal of attention. However, there is a general lack of understanding on why adversarial examples arise; whether they originate due to inherent properties of data or due…

Cited by 177Source ↗PDF ↗code ↗Scholar ↗
2017

Approximation and Convergence Properties of Generative Adversarial Learning

Shuang Liu, Olivier Bousquet, Kamalika Chaudhuri
NeurIPS 2017spotlight

Generative adversarial networks (GAN) approximate a target data distribution by jointly optimizing an objective function through a "two-player game" between a generator and a discriminator. Despite their empirical success, however, two very basic questions on how well they can approximate the targe…

Cited by 158Source ↗PDF ↗Scholar ↗
2017

Renyi Differential Privacy Mechanisms for Posterior Sampling

Joseph Geumlek, Shuang Song, Kamalika Chaudhuri
NeurIPS 2017poster

With the newly proposed privacy definition of Rényi Differential Privacy (RDP) in (Mironov, 2017), we re-examine the inherent privacy of releasing a single sample from a posterior distribution. We exploit the impact of the prior distribution in mitigating the influence of individual data points. In…

Cited by 89Source ↗PDF ↗code ↗Scholar ↗
2015

Convergence Rates of Active Learning for Maximum Likelihood Estimation

Kamalika Chaudhuri, Sham M. Kakade, Praneeth Netrapalli, Sujay Sanghavi
NeurIPS 2015poster

An active learner is given a class of models, a large set of unlabeled examples, and the ability to interactively query labels of a subset of these examples; the goal of the learner is to learn a model in the class that fits the data well. Previous theoretical work has rigorously characterized label…

Cited by 90Source ↗PDF ↗Scholar ↗
2015

Spectral Learning of Large Structured HMMs for Comparative Epigenomics

Chicheng Zhang, Jimin Song, Kamalika Chaudhuri, Kevin Chen
NeurIPS 2015poster

We develop a latent variable model and an efficient spectral algorithm motivated by the recent emergence of very large data sets of chromatin marks from multiple human cell types. A natural model for chromatin data in one cell type is a Hidden Markov Model (HMM); we model the relationship between mu…

Cited by 4Source ↗PDF ↗Scholar ↗