Utilizing Second-Order Information in Noisy Information-Sharing Environments for Distributed Optimization

Abstract

Decentralized optimization aims to cooperatively solve a global finite-sum loss function, where each agent only possesses knowledge of its own local function. Real-world applications introduce challenges such as unstable channels and differential privacy concerns, necessitating the development of more robust algorithms. This paper proposes a framework that explores second-order information using two approaches: global Newton tracking and local Newton preconditioning. Furthermore, we adapt a generic convergence result for gradient tracking methods to demonstrate the almost sure convergence property of both schemes under specific conditions. To validate the effectiveness of the proposed algorithm, numerical experiments are conducted on synthetic and real datasets, showcasing its robustness and superior accuracy compared to existing first-order methods.