2026
ReMoE: Boosting Expert Reuse through Router Fine-Tuning in Memory-Constrained MoE LLM Inference
ICML 2026poster
Fine-grained Mixture-of-Experts (MoE) models sparsely activate a subset of parameters, significantly reducing computational costs while maintaining performance. However, in memory-constrained inference scenarios, only a small set of experts can be cached. Experts not in the cache must be fetched fro…