2026
SERE: Similarity-based Expert Re-routing for Efficient Batch Decoding in MoE Models
ICLR 2026poster
Mixture-of-Experts (MoE) architectures employ sparse activation to deliver faster training and inference with higher accuracy than dense LLMs. However, in production serving, MoE models require batch inference to optimize hardware efficiency, which may cause excessive expert activation and thus slow…