Xinlai Kang

Breaking the Echo Chamber: A Dynamic Ensemble Pruning Perspective on MoE

We introduce Mahalanobis-Pruned Mixture-of-Experts (MP-MoE), a novel routing framework that approaches expert selection from the perspective of ensemble pruning. Existing Mixture-of-Experts (MoE) routing strategies often suffer from representation collapse due to greedy top-k selection mechanisms or…