AISTATS 2022poster27 citations
Conditionally Gaussian PAC-Bayes
Eugenio Clerico, George Deligiannidis, Arnaud Doucet
Abstract
Recent studies have empirically investigated different methods to train stochastic neural networks on a classification task by optimising a PAC-Bayesian bound via stochastic gradient descent. Most of these procedures need to replace the misclassification error with a surrogate loss, leading to a mismatch between the optimisation objective and the actual generalisation bound. The present paper proposes a novel training algorithm that optimises the PAC-Bayesian bound, without relying on any surrogate loss. Empirical results show that this approach outperforms currently available PAC-Bayesian training methods.
BibTeX
@InProceedings{pmlr-v151-clerico22a,
title = { Conditionally Gaussian PAC-Bayes },
author = {Clerico, Eugenio and Deligiannidis, George and Doucet, Arnaud},
booktitle = {Proceedings of The 25th International Conference on Artificial Intelligence and Statistics},
pages = {2311--2329},
year = {2022},
editor = {Camps-Valls, Gustau and Ruiz, Francisco J. R. and Valera, Isabel},
volume = {151},
series = {Proceedings of Machine Learning Research},
month = {28--30 Mar},
publisher = {PMLR},
pdf = {https://proceedings.mlr.press/v151/clerico22a/clerico22a.pdf},
url = {https://proceedings.mlr.press/v151/clerico22a.html},
abstract = { Recent studies have empirically investigated different methods to train stochastic neural networks on a classification task by optimising a PAC-Bayesian bound via stochastic gradient descent. Most of these procedures need to replace the misclassification error with a surrogate loss, leading to a mismatch between the optimisation objective and the actual generalisation bound. The present paper proposes a novel training algorithm that optimises the PAC-Bayesian bound, without relying on any surrogate loss. Empirical results show that this approach outperforms currently available PAC-Bayesian training methods. }
}