PAC Confidence Sets for Deep Neural Networks via Calibrated Prediction

Sangdon Park; Osbert Bastani; Nikolai Matni; Insup Lee

ICLR 2020poster76 citations

PAC Confidence Sets for Deep Neural Networks via Calibrated Prediction

Sangdon Park, Osbert Bastani, Nikolai Matni, Insup Lee

Abstract

We propose an algorithm combining calibrated prediction and generalization bounds from learning theory to construct confidence sets for deep neural networks with PAC guarantees---i.e., the confidence set for a given input contains the true label with high probability. We demonstrate how our approach can be used to construct PAC confidence sets on ResNet for ImageNet, a visual object tracking model, and a dynamics model for the half-cheetah reinforcement learning problem.

PACconfidence setsclassificationregressionreinforcement learning

BibTeX

@inproceedings{
Park2020PAC,
title={PAC Confidence Sets for Deep Neural Networks via Calibrated Prediction},
author={Sangdon Park and Osbert Bastani and Nikolai Matni and Insup Lee},
booktitle={International Conference on Learning Representations},
year={2020},
url={https://openreview.net/forum?id=BJxVI04YvB}
}