Pseudo-Labeling for Enhanced User Privacy in Approximate Machine Unlearning

Abstract

This paper presents an effective pseudo-labeling method for machine unlearning, focusing on protecting user privacy. Recent research predominantly focused on modifying the weights of pre-trained models to approximate exact unlearning. We study the impact of pseudo-labeling methods and propose two novel algorithms for approximate unlearning. Our approach aims to imitate the inference errors and behaviors that arise from exact unlearning of a subset of the target data to be forgotten. To identify the data within the forget set where inferences tend to shift in the wrong direction, we conduct comparative analyses using a pre-trained model, a simple fine-tuning model, and a simple scratch model. We use identified data along with incorrect labels derived from the wrong direction for pseudo-labeling. Experimental evaluations show that our method achieves superior performance in protecting user privacy, demonstrating enhanced forgetting rates and particularly robust defense against membership inference attack.