Learning Curve Prediction with Bayesian Neural Networks

Aaron Klein; Stefan Falkner; Jost Tobias Springenberg; Frank Hutter

ICLR 2017poster301 citations

Learning Curve Prediction with Bayesian Neural Networks

Aaron Klein, Stefan Falkner, Jost Tobias Springenberg, Frank Hutter

Abstract

Different neural network architectures, hyperparameters and training protocols lead to different performances as a function of time. Human experts routinely inspect the resulting learning curves to quickly terminate runs with poor hyperparameter settings and thereby considerably speed up manual hyperparameter optimization. Exploiting the same information in automatic Bayesian hyperparameter optimization requires a probabilistic model of learning curves across hyperparameter settings. Here, we study the use of Bayesian neural networks for this purpose and improve their performance by a specialized learning curve layer.

Deep learningApplications

BibTeX

@inproceedings{
klein2017learning,
title={Learning Curve Prediction with Bayesian Neural Networks},
author={Aaron Klein and Stefan Falkner and Jost Tobias Springenberg and Frank Hutter},
booktitle={International Conference on Learning Representations},
year={2017},
url={https://openreview.net/forum?id=S11KBYclx}
}