Kernel Interpolation for Scalable Online Gaussian Processes
Samuel Stanton, Wesley Maddox, Ian Delbridge, Andrew Gordon Wilson
Abstract
Gaussian processes (GPs) provide a gold standard for performance in online settings, such as sample-efficient control and black box optimization, where we need to update a posterior distribution as we acquire data in a sequential online setting. However, updating a GP posterior to accommodate even a single new observation after having observed $n$ points incurs at least $\mathcal{O}(n)$ computations in the exact setting. We show how to use structured kernel interpolation to efficiently reuse computations for constant-time $\mathcal{O}(1)$ online updates with respect to the number of points $n$, while retaining exact inference. We demonstrate the promise of our approach in a range of online regression and classification settings, Bayesian optimization, and active sampling to reduce error in malaria incidence forecasting.
BibTeX
@InProceedings{pmlr-v130-stanton21a,
title = { Kernel Interpolation for Scalable Online Gaussian Processes },
author = {Stanton, Samuel and Maddox, Wesley and Delbridge, Ian and Gordon Wilson, Andrew},
booktitle = {Proceedings of The 24th International Conference on Artificial Intelligence and Statistics},
pages = {3133--3141},
year = {2021},
editor = {Banerjee, Arindam and Fukumizu, Kenji},
volume = {130},
series = {Proceedings of Machine Learning Research},
month = {13--15 Apr},
publisher = {PMLR},
pdf = {http://proceedings.mlr.press/v130/stanton21a/stanton21a.pdf},
url = {https://proceedings.mlr.press/v130/stanton21a.html},
abstract = { Gaussian processes (GPs) provide a gold standard for performance in online settings, such as sample-efficient control and black box optimization, where we need to update a posterior distribution as we acquire data in a sequential online setting. However, updating a GP posterior to accommodate even a single new observation after having observed $n$ points incurs at least $\mathcal{O}(n)$ computations in the exact setting. We show how to use structured kernel interpolation to efficiently reuse computations for constant-time $\mathcal{O}(1)$ online updates with respect to the number of points $n$, while retaining exact inference. We demonstrate the promise of our approach in a range of online regression and classification settings, Bayesian optimization, and active sampling to reduce error in malaria incidence forecasting. }
}