A Lightweight Mixture-of-Experts Neural Machine Translation Model with Stage-wise Training Strategy
Dealing with language heterogeneity has always been one of the challenges in neural machine translation (NMT).The idea of using mixture-of-experts (MoE) naturally excels in addressing this issue by employing different experts to take responsibility for different problems.However, the parameter-ineff…