ICLR 2017poster2223 citations
Deep Variational Information Bottleneck
Alexander A. Alemi, Ian Fischer, Joshua V. Dillon, Kevin Murphy
Abstract
We present a variational approximation to the information bottleneck of Tishby et al. (1999). This variational approach allows us to parameterize the information bottleneck model using a neural network and leverage the reparameterization trick for efficient training. We call this method “Deep Variational Information Bottleneck”, or Deep VIB. We show that models trained with the VIB objective outperform those that are trained with other forms of regularization, in terms of generalization performance and robustness to adversarial attack.
TheoryComputer visionDeep learningSupervised Learning
BibTeX
@inproceedings{
alemi2017deep,
title={Deep Variational Information Bottleneck},
author={Alexander A. Alemi and Ian Fischer and Joshua V. Dillon and Kevin Murphy},
booktitle={International Conference on Learning Representations},
year={2017},
url={https://openreview.net/forum?id=HyxQzBceg}
}