Shanhong yu

RaGEP: Rank-aware Geometric Expert Pruning for Mixture-of-Experts Language Models

Sparse Mixture-of-Experts (MoE) architectures scale model capacity efficiently but suffer from massive static parameter footprints, creating significant deployment burdens on memory-constrained hardware. Existing post-training pruning methods often rely on scalar statistics, ignoring the representat…