2026
Revisiting Efficiency–Accuracy Scaling in Mixture-of-Experts Architectures
ICML 2026poster
Mixture-of-Experts (MoEs) have become a central component of many state-of-the-art open-source and proprietary large language models. Despite their widespread adoption, it remains unclear how close existing MoE architectures are to optimal with respect to inference cost, as measured by accuracy per …