ICML 2018oral374 citations
High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approach
Tim Pearce, Alexandra Brintrup, Mohamed Zaki, Andy Neely
Abstract
This paper considers the generation of prediction intervals (PIs) by neural networks for quantifying uncertainty in regression tasks. It is axiomatic that high-quality PIs should be as narrow as possible, whilst capturing a specified portion of data. We derive a loss function directly from this axiom that requires no distributional assumption. We show how its form derives from a likelihood principle, that it can be used with gradient descent, and that model uncertainty is accounted for in ensembled form. Benchmark experiments show the method outperforms current state-of-the-art uncertainty quantification methods, reducing average PI width by over 10%.
BibTeX
@InProceedings{pmlr-v80-pearce18a,
title = {High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approach},
author = {Pearce, Tim and Brintrup, Alexandra and Zaki, Mohamed and Neely, Andy},
booktitle = {Proceedings of the 35th International Conference on Machine Learning},
pages = {4075--4084},
year = {2018},
editor = {Dy, Jennifer and Krause, Andreas},
volume = {80},
series = {Proceedings of Machine Learning Research},
month = {10--15 Jul},
publisher = {PMLR},
pdf = {http://proceedings.mlr.press/v80/pearce18a/pearce18a.pdf},
url = {https://proceedings.mlr.press/v80/pearce18a.html},
abstract = {This paper considers the generation of prediction intervals (PIs) by neural networks for quantifying uncertainty in regression tasks. It is axiomatic that high-quality PIs should be as narrow as possible, whilst capturing a specified portion of data. We derive a loss function directly from this axiom that requires no distributional assumption. We show how its form derives from a likelihood principle, that it can be used with gradient descent, and that model uncertainty is accounted for in ensembled form. Benchmark experiments show the method outperforms current state-of-the-art uncertainty quantification methods, reducing average PI width by over 10%.}
}