UQ-Guided Hyperparameter Optimization for Iterative Learners
Jiesong Liu, Feng Zhang, Jiawei Guan, Xipeng Shen
Abstract
Hyperparameter Optimization (HPO) plays a pivotal role in unleashing the potential of iterative machine learning models. This paper addresses a crucial aspect that has largely been overlooked in HPO: the impact of uncertainty in ML model training. The paper introduces the concept of uncertainty-aware HPO and presents a novel approach called the UQ-guided scheme for quantifying uncertainty. This scheme offers a principled and versatile method to empower HPO techniques in handling model uncertainty during their exploration of the candidate space. By constructing a probabilistic model and implementing probability-driven candidate selection and budget allocation, this approach enhances the quality of the resulting model hyperparameters. It achieves a notable performance improvement of over 50\% in terms of accuracy regret and exploration time.
BibTeX
@inproceedings{
liu2024uqguided,
title={{UQ}-Guided Hyperparameter Optimization for Iterative Learners},
author={Jiesong Liu and Feng Zhang and Jiawei Guan and Xipeng Shen},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=k9uZfaeerK}
}