Learning Hierarchical Priors in VAEs
Alexej Klushyn, Nutan Chen, Richard Kurle, Botond Cseke, Patrick van der Smagt
Abstract
We propose to learn a hierarchical prior in the context of variational autoencoders to avoid the over-regularisation resulting from a standard normal prior distribution. To incentivise an informative latent representation of the data, we formulate the learning problem as a constrained optimisation problem by extending the Taming VAEs framework to two-level hierarchical models. We introduce a graph-based interpolation method, which shows that the topology of the learned latent representation corresponds to the topology of the data manifold---and present several examples, where desired properties of latent representation such as smoothness and simple explanatory factors are learned by the prior.
BibTeX
@inproceedings{NEURIPS2019_7d12b66d,
author = {Klushyn, Alexej and Chen, Nutan and Kurle, Richard and Cseke, Botond and van der Smagt, Patrick},
booktitle = {Advances in Neural Information Processing Systems},
editor = {H. Wallach and H. Larochelle and A. Beygelzimer and F. d\textquotesingle Alch\'{e}-Buc and E. Fox and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {Learning Hierarchical Priors in VAEs},
url = {https://proceedings.neurips.cc/paper_files/paper/2019/file/7d12b66d3df6af8d429c1a357d8b9e1a-Paper.pdf},
volume = {32},
year = {2019}
}