Charles-Étienne Joseph

$\mu$LO: Compute-Efficient Meta-Generalization of Learned Optimizers

Learned optimizers (LOs) have the potential to significantly reduce the wall-clock training time of neural networks. However, they can struggle to optimize unseen tasks (*meta-generalize*), especially when training networks wider than those seen during meta-training. To address this, we derive the M…