LossControl: Defending Membership Inference Attacks by Controlling the Loss

Abstract

Machine learning models are vulnerable to membership inference attacks (MIAs), where adversaries attempt to predict whether specific samples are part of the model’s training set. Previous studies have demonstrated a strong correlation between the distinguishability of training and testing loss distributions and the model’s susceptibility to MIAs. Motivated by existing results, we propose a novel training framework called LossControl, which focuses on manipulating loss to mitigate privacy leaks. In LossControl, we first utilize Soft-label Training to replace the general learning process, which facilitates model training while improving generalization. Next, we monitor overfitting samples during the training process and prevent further loss reduction by applying our designed Loss Ascent to these samples without sacrificing model performance. Through extensive evaluations across four diverse datasets (including images, medical data, and transaction records), our method consistently outperforms defense mechanisms against state-of-the-art attacks and achieves optimal model performance in most experiments, demonstrating LossControl’s superior resilience against MIAs and its ability to strike an unparalleled balance between privacy and utility.