Towards Constituting Mathematical Structures for Learning to Optimize
Jialin Liu, Xiaohan Chen, Zhangyang Wang, Wotao Yin, HanQin Cai
Abstract
Learning to Optimize (L2O), a technique that utilizes machine learning to learn an optimization algorithm automatically from data, has gained arising attention in recent years. A generic L2O approach parameterizes the iterative update rule and learns the update direction as a black-box network. While the generic approach is widely applicable, the learned model can overfit and may not generalize well to out-of-distribution test sets. In this paper, we derive the basic mathematical conditions that successful update rules commonly satisfy. Consequently, we propose a novel L2O model with a mathematics-inspired structure that is broadly applicable and generalized well to out-of-distribution problems. Numerical simulations validate our theoretical findings and demonstrate the superior empirical performance of the proposed L2O model.
BibTeX
@inproceedings{icml2023_towardsconstitut,
title = {Towards Constituting Mathematical Structures for Learning to Optimize},
author = {Jialin Liu and Xiaohan Chen and Zhangyang Wang and Wotao Yin and HanQin Cai},
booktitle = {ICML 2023},
year = {2023}
}