Federated PAC-Bayesian Learning on Non-IID Data

Zihao Zhao; Yang Liu; Wenbo Ding; Xiao-Ping Zhang

ICASSP 2024accepted0 citations

Federated PAC-Bayesian Learning on Non-IID Data

Zihao Zhao, Yang Liu, Wenbo Ding, Xiao-Ping Zhang

Abstract

Existing research has either adapted the Probably Approximately Correct (PAC) Bayesian framework for federated learning (FL) or used information-theoretic PAC-Bayesian bounds while introducing their theorems, but few consider the non-IID challenges in FL. Our work presents the first non-vacuous federated PAC-Bayesian bound tailored for non-IID local data. This bound assumes unique prior knowledge for each client and variable aggregation weights. We also introduce an objective function and an innovative Gibbs-based algorithm for the optimization of the derived bound. The results are validated on real-world datasets.

BibTeX

@inproceedings{icassp2024_federatedpacbaye,
  title = {Federated PAC-Bayesian Learning on Non-IID Data},
  author = {Zihao Zhao and Yang Liu and Wenbo Ding and Xiao-Ping Zhang},
  booktitle = {ICASSP 2024},
  year = {2024}
}