2026
From Muon to Gluon: Bridging Theory and Practice of LMO-based Optimizers for LLMs
ICML 2026poster
Recent developments in deep learning optimization have brought about radically new algorithms based on the Linear Minimization Oracle (LMO) framework, such as Muon and Scion. After over a decade of Adam's dominance, these LMO-based methods are emerging as viable replacements, offering several practi…