Stephan Sloth Lorenzen — accepted AI-conference papers

2022

Information Bottleneck: Exact Analysis of (Quantized) Neural Networks

Stephan Sloth Lorenzen, Christian Igel, Mads Nielsen

ICLR 2022poster

The information bottleneck (IB) principle has been suggested as a way to analyze deep neural networks. The learning dynamics are studied by inspecting the mutual information (MI) between the hidden layers and the input and output. Notably, separate fitting and compression phases during training hav…

Cited by 19 Source ↗PDF ↗code ↗Scholar ↗

2021

Chebyshev-Cantelli PAC-Bayes-Bennett Inequality for the Weighted Majority Vote

Yi-Shan Wu, Andres R Masegosa, Stephan Sloth Lorenzen, Christian Igel, Yevgeny Seldin

NeurIPS 2021poster

We present a new second-order oracle bound for the expected risk of a weighted majority vote. The bound is based on a novel parametric form of the Chebyshev-Cantelli inequality (a.k.a. one-sided Chebyshev’s), which is amenable to efficient minimization. The new form resolves the optimization challen…

Cited by 19 Source ↗PDF ↗code ↗Scholar ↗