Yuchao Chen

MoE Jetpack: From Dense Checkpoints to Adaptive Mixture of Experts for Vision Tasks

The sparsely activated mixture of experts (MoE) model presents an effective alternative to densely activated (dense) models, combining improved accuracy with computational efficiency. However, training MoE models from scratch requires extensive data and computational resources, a challenge that limi…