2026
$\phi$-Balancing for Mixture-of-Experts Training
ICML 2026poster
Mixture-of-Experts (MoE) models rely on balanced expert utilization to fully realize their scalability. However, existing load-balancing methods are largely heuristic and operate on mini-batch assignment statistics, introducing bias relative to population-level objectives. We propose $\phi$-balancin…