Personalized Local Differentially Private Federated Learning with Adaptive Client Sampling

Abstract

Differentially Private Federated Learning (DP-FL) is a promising paradigm for training models on large-scale decentralized data under Differential Privacy (DP) guarantees which confronts two challenges: 1) providing a privacy guarantee without sacrificing model performance; 2) tackling system heterogeneity and data heterogeneity under DP. Recent works on DP-FL have focused on uniform client sampling and privacy settings, which neglects the impact of client sampling on the trade-offs between utility, communication, and personalized privacy. This paper proposes a novel Adaptive Client Sampling algorithm for Personalized Local Differentially Private Federated Learning to address these issues. We derive a new convergence bound for non-convex objectives with personalized differential privacy and arbitrary client sampling. We also analyze PLDP with client sampling, maintaining the same level of privacy guarantee with a smaller noise scale. Based on the bound and analysis, we establish the relation between client sampling, privacy bound, and utility bound, resulting in optimization problems for non-convex bound minimization. Simulation and prototype results using MNIST and EMNIST datasets demonstrate that our algorithm is superior to existing baselines.