AIConfPaper
← Search

Sanjay Kariyappa

7 accepted papers

2024

Progressive Inference: Explaining Decoder-Only Sequence Classification Models Using Intermediate Predictions

Sanjay Kariyappa, Freddy Lecue, Saumitra Mishra, Christopher Pond, Daniele Magazzeni, Manuela Veloso
ICML 2024poster

This paper proposes Progressive inference--a framework to explain the predictions of decoder-only transformer models trained to perform sequence classification tasks. Our work is based on the insight that the classification head of a decoder-only model can be used to make intermediate predictions by…

Cited by 1Source ↗PDF ↗Scholar ↗
2024

SHAP@k: Efficient and Probably Approximately Correct (PAC) Identification of Top-K Features

Sanjay Kariyappa, Leonidas Tsepenekas, Freddy Lécué, Daniele Magazzeni
AAAI 2024technical

The SHAP framework provides a principled method to explain the predictions of a model by computing feature importance. Motivated by applications in finance, we introduce the Top-k Identification Problem (TkIP) (and its ordered variant TkIP- O), where the objective is to identify the subset (or order…

Cited by 3Source ↗PDF ↗Scholar ↗
2023

Bounding the Invertibility of Privacy-preserving Instance Encoding using Fisher Information

Kiwan Maeng, Chuan Guo, Sanjay Kariyappa, G. Edward Suh
NeurIPS 2023poster

Privacy-preserving instance encoding aims to encode raw data into feature vectors without revealing their privacy-sensitive information. When designed properly, these encodings can be used for downstream ML applications such as training and inference with limited privacy risk. However, the vast majo…

Cited by 11Source ↗PDF ↗Scholar ↗
2023

Cocktail Party Attack: Breaking Aggregation-Based Privacy in Federated Learning Using Independent Component Analysis

Sanjay Kariyappa, Chuan Guo, Kiwan Maeng, Wenjie Xiong, G. Edward Suh, Moinuddin K Qureshi +1
ICML 2023poster

Federated learning (FL) aims to perform privacy-preserving machine learning on distributed data held by multiple data owners. To this end, FL requires the data owners to perform training locally and share the gradients or weight updates (instead of the private inputs) with the central server, which…

Cited by 36Source ↗PDF ↗Scholar ↗
2021

MAZE: Data-Free Model Stealing Attack Using Zeroth-Order Gradient Estimation

Sanjay Kariyappa, Atul Prakash, Moinuddin K Qureshi
CVPR 2021poster

High quality Machine Learning (ML) models are often considered valuable intellectual property by companies. Model Stealing (MS) attacks allow an adversary with black-box access to a ML model to replicate its functionality by training a clone model using the predictions of the target model for differ…

Cited by 203PDF ↗code ↗Scholar ↗
2021

Protecting DNNs from Theft using an Ensemble of Diverse Models

Sanjay Kariyappa, Atul Prakash, Moinuddin K Qureshi
ICLR 2021poster

Several recent works have demonstrated highly effective model stealing (MS) attacks on Deep Neural Networks (DNNs) in black-box settings, even when the training data is unavailable. These attacks typically use some form of Out of Distribution (OOD) data to query the target model and use the predicti…

Cited by 37Source ↗PDF ↗Scholar ↗