Yuncheng Song

Self-Evolution Knowledge Distillation for LLM-based Machine Translation

Knowledge distillation (KD) has shown great promise in transferring knowledge from larger teacher models to smaller student models. However, existing KD strategies for large language models often minimize output distributions between student and teacher models indiscriminately for each token. This o…