Weifeng Yang

Schur-A*: Layer-wise Optimal Expert Pruning for Sparse MoEs via Schur-Complement Guided A* Search

Sparse Mixture-of-Experts (MoE) language models enable conditional computation but face deployment challenges due to the "memory wall": while few experts are activated per token, the entire model must reside in memory. Existing expert pruning methods primarily rely on independent ranking, failing to…