Distributed Gaussian Process Hyperparameter Optimization for Multi-Agent Systems

Abstract

Gaussian Process (GP) is a flexible non-parametric method which has a wide variety of applications e.g., field estimation using multi-agent systems. However, the training of the hyperparameters suffers from high computational complexity. Recently, distributed hyperparameter optimization with proximal gradients has been proposed to reduce complexity, however only for a network with a central station. In this work, exploiting edge-based constraints, we propose two fully-distributed algorithms pxADMM <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">fd</inf> and pxADMM <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">fd,fast</inf> for a network of multi-agent systems, which do not rely on a central station. In addition, asynchronous versions of the algorithms are also proposed to reduce the synchronization overhead in heterogeneous networks. Simulations are conducted for a field estimation problem, using both artificial, and real-world datasets, which show that the proposed fully-distributed algorithms successfully converge, at the cost of an increased number of iterations.