Personalization Improves Privacy-Accuracy Tradeoffs in Federated Learning
Alberto Bietti, Chen-Yu Wei, Miroslav Dudik, John Langford, Steven Wu
Abstract
Large-scale machine learning systems often involve data distributed across a collection of users. Federated learning algorithms leverage this structure by communicating model updates to a central server, rather than entire datasets. In this paper, we study stochastic optimization algorithms for a personalized federated learning setting involving local and global models subject to user-level (joint) differential privacy. While learning a private global model induces a cost of privacy, local learning is perfectly private. We provide generalization guarantees showing that coordinating local learning with private centralized learning yields a generically useful and improved tradeoff between accuracy and privacy. We illustrate our theoretical results with experiments on synthetic and real-world datasets.
BibTeX
@InProceedings{pmlr-v162-bietti22a,
title = {Personalization Improves Privacy-Accuracy Tradeoffs in Federated Learning},
author = {Bietti, Alberto and Wei, Chen-Yu and Dudik, Miroslav and Langford, John and Wu, Steven},
booktitle = {Proceedings of the 39th International Conference on Machine Learning},
pages = {1945--1962},
year = {2022},
editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan},
volume = {162},
series = {Proceedings of Machine Learning Research},
month = {17--23 Jul},
publisher = {PMLR},
pdf = {https://proceedings.mlr.press/v162/bietti22a/bietti22a.pdf},
url = {https://proceedings.mlr.press/v162/bietti22a.html},
abstract = {Large-scale machine learning systems often involve data distributed across a collection of users. Federated learning algorithms leverage this structure by communicating model updates to a central server, rather than entire datasets. In this paper, we study stochastic optimization algorithms for a personalized federated learning setting involving local and global models subject to user-level (joint) differential privacy. While learning a private global model induces a cost of privacy, local learning is perfectly private. We provide generalization guarantees showing that coordinating local learning with private centralized learning yields a generically useful and improved tradeoff between accuracy and privacy. We illustrate our theoretical results with experiments on synthetic and real-world datasets.}
}