Unveiling the Pruning Risks on Privacy Vulnerabilities of Deep Neural Networks

Abstract

Large-scale deep neural networks (DNNs), such as large language models, have gained immense popularity due to their outstanding performance across various tasks. However, their application in resource-constrained scenarios faces significant challenges due to the high computational costs and memory usage of these models during inference. Model pruning emerges as a viable technique to mitigate these limitations by reducing the computational complexity of deep models. While existing research primarily focuses on maximizing inference efficiency without compromising accuracy, the privacy implications of pruning techniques remain largely unexplored. In this work, we systematically investigate the impact of popular pruning techniques on the privacy vulnerabilities of DNNs. We begin by applying common pruning schemes to various DNNs and evaluating their privacy risks, both before and after pruning, using model inversion attacks. We then analyze how the pruning rate and granularity affect these privacy vulnerabilities. Moreover, we conduct experiments on both original and pruned models equipped with defenses to confirm that the increase in privacy risks following pruning is not merely coincidental. Finally, we offer guidelines for the careful application of pruning techniques. Our findings serve as a cautionary note, highlighting the inherent privacy risks associated with current pruning schemes and providing valuable insights for developing pruning methods that are both efficient and secure.