2024
Align-to-Distill: Trainable Attention Alignment for Knowledge Distillation in Neural Machine Translation
COLING 2024main
The advent of scalable deep models and large datasets has improved the performance of Neural Machine Translation (NMT). Knowledge Distillation (KD) enhances efficiency by transferring knowledge from a teacher model to a more compact student model. However, KD approaches to Transformer architecture o…