2026
Unveiling Super Experts in Mixture-of-Experts Large Language Models
ICLR 2026poster
Leveraging the intrinsic importance differences among experts, recent research has explored expert-level compression techniques to enhance the efficiency of Mixture-of-Experts (MoE) large language models (LLMs). However, existing approaches often rely on empirical heuristics to identify critical exp…