A New Distribution on the Simplex with Auto-Encoding Applications
Andrew Stirn, Tony Jebara, David Knowles
Abstract
We construct a new distribution for the simplex using the Kumaraswamy distribution and an ordered stick-breaking process. We explore and develop the theoretical properties of this new distribution and prove that it exhibits symmetry (exchangeability) under the same conditions as the well-known Dirichlet. Like the Dirichlet, the new distribution is adept at capturing sparsity but, unlike the Dirichlet, has an exact and closed form reparameterization--making it well suited for deep variational Bayesian modeling. We demonstrate the distribution's utility in a variety of semi-supervised auto-encoding tasks. In all cases, the resulting models achieve competitive performance commensurate with their simplicity, use of explicit probability models, and abstinence from adversarial training.
BibTeX
@inproceedings{NEURIPS2019_43207fd5,
author = {Stirn, Andrew and Jebara, Tony and Knowles, David},
booktitle = {Advances in Neural Information Processing Systems},
editor = {H. Wallach and H. Larochelle and A. Beygelzimer and F. d\textquotesingle Alch\'{e}-Buc and E. Fox and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {A New Distribution on the Simplex with Auto-Encoding Applications},
url = {https://proceedings.neurips.cc/paper_files/paper/2019/file/43207fd5e34f87c48d584fc5c11befb8-Paper.pdf},
volume = {32},
year = {2019}
}