Artemy Kolchinsky — accepted AI-conference papers

2019

Caveats for information bottleneck in deterministic scenarios

Artemy Kolchinsky, Brendan D. Tracey, Steven Van Kuyk

ICLR 2019poster

Information bottleneck (IB) is a method for extracting information from one random variable X that is relevant for predicting another random variable Y. To do so, IB identifies an intermediate "bottleneck" variable T that has low mutual information I(X;T) and high mutual information I(Y;T). The "IB…

Cited by 101 Source ↗PDF ↗code ↗Scholar ↗

2018

On the Information Bottleneck Theory of Deep Learning

Andrew Michael Saxe, Yamini Bansal, Joel Dapello, Madhu Advani, Artemy Kolchinsky, Brendan Daniel Tracey +1

ICLR 2018poster

The practical successes of deep neural networks have not been matched by theoretical progress that satisfyingly explains their behavior. In this work, we study the information bottleneck (IB) theory of deep learning, which makes three specific claims: first, that deep networks undergo two distinct p…

Cited by 723 Source ↗PDF ↗code ↗Scholar ↗