2019
Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functions
ICLR 2019poster
Hyperparameter optimization can be formulated as a bilevel optimization problem, where the optimal parameters on the training set depend on the hyperparameters. We aim to adapt regularization hyperparameters for neural networks by fitting compact approximations to the best-response function, which m…