2026
Natural Hypergradient Descent: Algorithm Design, Convergence Analysis, and Parallel Implementation
ICML 2026poster
In this work, we propose *Natural Hypergradient Descent* (NHGD), a new method for solving bilevel optimization problems. To address the computational bottleneck in hypergradient estimation—namely, the need to compute or approximate Hessian inverses—we exploit the statistical structure of the inner o…