Personalized Federated Training of Diffusion Models with Privacy Guarantees
Kumar Kshitij Patel, Bingqing Jiang, A F M Mahfuzul Kabir, Weitong Zhang, Difan Zou, Lingxiao Wang
Abstract
We propose a federated framework for training diffusion models on decentralized and private datasets. The method learns a shared generative model alongside personalized client models, allowing clients to benefit from cross-client structure while ensuring that the shared model cannot reproduce any client's data on its own. We provide formal differential privacy guarantees for each client and establish utility bounds for conditional generation under a Gaussian mixture model, showing that collaboration improves sample quality relative to private non-collaborative training. Experiments on CIFAR-10, Colorized MNIST, and CelebA support these results: the method generates high-fidelity samples, improves performance on minority and underrepresented classes, and maintains strong protection against membership inference, memorization, and reconstruction attacks.
BibTeX
@inproceedings{cvpr2026_personalizedfede,
title = {Personalized Federated Training of Diffusion Models with Privacy Guarantees},
author = {Kumar Kshitij Patel and Bingqing Jiang and A F M Mahfuzul Kabir and Weitong Zhang and Difan Zou and Lingxiao Wang},
booktitle = {CVPR 2026},
year = {2026}
}