Local Entropy Search over Descent Sequences for Bayesian Optimization

Abstract

Searching large and highly complex design spaces for a global optimum can be infeasible and unnecessary. A practical alternative is to iteratively refine the neighborhood of an initial design using local optimization methods such as gradient descent. We propose local entropy search (LES), a Bayesian optimization paradigm that explicitly targets the solutions reachable by the descent sequences of arbitrary iterative optimizers. The algorithm propagates the posterior belief over the objective through the optimizer, yielding a probability distribution over descent sequences. It then selects the next evaluation by maximizing mutual information with that distribution, using a practical combination of analytic entropy calculations and Monte-Carlo sampling of descent sequences. Empirical results on high-complexity synthetic objectives and benchmark problems show that LES achieves strong sample efficiency compared to existing local and global Bayesian optimization methods.