Junwei Cui

Mining Tensor/Neuron-Level Sparsity to Maximize Mixture-of-Experts Potential in Post-Training and Inference

Mixture of Experts (MoE) has emerged as a mainstream architecture for Large Language Models (LLMs), balancing computational efficiency with model scalability. While prior work has explored increasing tensor-level sparsity via finer-grained expert configurations during pre-training, we identify signi…