Byzantine-Robust and Communication-Efficient Distributed Non-Convex Learning Over Non-IID Data

Xuechao He; Heng Zhu; Qing Ling

ICASSP 2022accepted0 citations

Byzantine-Robust and Communication-Efficient Distributed Non-Convex Learning Over Non-IID Data

Abstract

Motivated by the emerging federated learning applications, we jointly consider the problems of Byzantine-robustness and communication efficiency in distributed non-convex learning over non-IID data. We propose a compressed robust stochastic model aggregation (CRSA) method, which applies the idea of robust stochastic model aggregation to achieve Byzantine-robustness over non-IID data, while compresses the transmitted messages so as to achieve communication efficiency. Utilizing the tools of Moreau envelope and proximal point projection, we establish the convergence of C-RSA for distributed non-convex learning problems. Numerical experiments on training a large-scale neural network demonstrate the effectiveness of the proposed C-RSA method.

BibTeX

@inproceedings{icassp2022_byzantinerobusta,
  title = {Byzantine-Robust and Communication-Efficient Distributed Non-Convex Learning Over Non-IID Data},
  author = {Xuechao He and Heng Zhu and Qing Ling},
  booktitle = {ICASSP 2022},
  year = {2022}
}