Optimal Quantisation of Probability Measures Using Maximum Mean Discrepancy
Onur Teymur, Jackson Gorham, Marina Riabiz, Chris Oates
Abstract
Several researchers have proposed minimisation of maximum mean discrepancy (MMD) as a method to quantise probability measures, i.e., to approximate a distribution by a representative point set. We consider sequential algorithms that greedily minimise MMD over a discrete candidate set. We propose a novel non-myopic algorithm and, in order to both improve statistical efficiency and reduce computational cost, we investigate a variant that applies this technique to a mini-batch of the candidate set at each iteration. When the candidate points are sampled from the target, the consistency of these new algorithms—and their mini-batch variants—is established. We demonstrate the algorithms on a range of important computational problems, including optimisation of nodes in Bayesian cubature and the thinning of Markov chain output.
BibTeX
@InProceedings{pmlr-v130-teymur21a,
title = { Optimal Quantisation of Probability Measures Using Maximum Mean Discrepancy },
author = {Teymur, Onur and Gorham, Jackson and Riabiz, Marina and Oates, Chris},
booktitle = {Proceedings of The 24th International Conference on Artificial Intelligence and Statistics},
pages = {1027--1035},
year = {2021},
editor = {Banerjee, Arindam and Fukumizu, Kenji},
volume = {130},
series = {Proceedings of Machine Learning Research},
month = {13--15 Apr},
publisher = {PMLR},
pdf = {http://proceedings.mlr.press/v130/teymur21a/teymur21a.pdf},
url = {https://proceedings.mlr.press/v130/teymur21a.html},
abstract = { Several researchers have proposed minimisation of maximum mean discrepancy (MMD) as a method to quantise probability measures, i.e., to approximate a distribution by a representative point set. We consider sequential algorithms that greedily minimise MMD over a discrete candidate set. We propose a novel non-myopic algorithm and, in order to both improve statistical efficiency and reduce computational cost, we investigate a variant that applies this technique to a mini-batch of the candidate set at each iteration. When the candidate points are sampled from the target, the consistency of these new algorithms—and their mini-batch variants—is established. We demonstrate the algorithms on a range of important computational problems, including optimisation of nodes in Bayesian cubature and the thinning of Markov chain output. }
}