Byzantine-resilient Distributed Large-scale Matrix Completion

Abstract

In this paper, we aim at completing a large-scale low-rank matrix over a distributed network, which is subject to Byzantine attacks. We consider solving a nonconvex matrix factorization model with the distributed successive over-relaxation (SOR) method, where the distributed workers compute their private matrices using their own training data and the public matrix sent by the master, while the master updates the public matrix through aggregating the private matrices sent by the workers. However, the Byzantine workers could deliberately send faulty messages to the master so as to bias the optimization process. To address this issue, we propose to replace the aggregation step in the distributed SOR method by several state-of-the-art robust ones: geometric median, median, Krum and h-Krum. We conduct numerical experiments on the Netflix dataset and demonstrate the effectiveness of the proposed robust aggregation strategies in handling Byzantine attacks.