AISTATS 2016poster114 citations
PAC-Bayesian Bounds based on the Rényi Divergence
Luc Bégin, Pascal Germain, François Laviolette, Jean-Francis Roy
Abstract
We propose a simplified proof process for PAC-Bayesian generalization bounds, that allows to divide the proof in four successive inequalities, easing the "customization" of PAC-Bayesian theorems. We also propose a family of PAC-Bayesian bounds based on the Rényi divergence between the prior and posterior distributions, whereas most PAC-Bayesian bounds are based on the Kullback-Leibler divergence. Finally, we present an empirical evaluation of the tightness of each inequality of the simplified proof, for both the classical PAC-Bayesian bounds and those based on the Rényi divergence.
BibTeX
@InProceedings{pmlr-v51-begin16,
title = {PAC-Bayesian Bounds based on the Rényi Divergence},
author = {Bégin, Luc and Germain, Pascal and Laviolette, François and Roy, Jean-Francis},
booktitle = {Proceedings of the 19th International Conference on Artificial Intelligence and Statistics},
pages = {435--444},
year = {2016},
editor = {Gretton, Arthur and Robert, Christian C.},
volume = {51},
series = {Proceedings of Machine Learning Research},
address = {Cadiz, Spain},
month = {09--11 May},
publisher = {PMLR},
pdf = {http://proceedings.mlr.press/v51/begin16.pdf},
url = {https://proceedings.mlr.press/v51/begin16.html},
abstract = {We propose a simplified proof process for PAC-Bayesian generalization bounds, that allows to divide the proof in four successive inequalities, easing the "customization" of PAC-Bayesian theorems. We also propose a family of PAC-Bayesian bounds based on the Rényi divergence between the prior and posterior distributions, whereas most PAC-Bayesian bounds are based on the Kullback-Leibler divergence. Finally, we present an empirical evaluation of the tightness of each inequality of the simplified proof, for both the classical PAC-Bayesian bounds and those based on the Rényi divergence.}
}