Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networks
Gaël Letarte, Pascal Germain, Benjamin Guedj, Francois Laviolette
Abstract
We present a comprehensive study of multilayer neural networks with binary activation, relying on the PAC-Bayesian theory. Our contributions are twofold: (i) we develop an end-to-end framework to train a binary activated deep neural network, (ii) we provide nonvacuous PAC-Bayesian generalization bounds for binary activated deep neural networks. Our results are obtained by minimizing the expected loss of an architecture-dependent aggregation of binary activated deep neural networks. Our analysis inherently overcomes the fact that binary activation function is non-differentiable. The performance of our approach is assessed on a thorough numerical experiment protocol on real-life datasets.
BibTeX
@inproceedings{NEURIPS2019_7ec3b3cf,
author = {Letarte, Ga\"{e}l and Germain, Pascal and Guedj, Benjamin and Laviolette, Francois},
booktitle = {Advances in Neural Information Processing Systems},
editor = {H. Wallach and H. Larochelle and A. Beygelzimer and F. d\textquotesingle Alch\'{e}-Buc and E. Fox and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networks},
url = {https://proceedings.neurips.cc/paper_files/paper/2019/file/7ec3b3cf674f4f1d23e9d30c89426cce-Paper.pdf},
volume = {32},
year = {2019}
}