2026
Heterogeneous Complementary Distillation
AAAI 2026technical
Knowledge distillation (KD) transfers the ``dark knowledge
2 accepted papers
Knowledge distillation (KD) transfers the ``dark knowledge
Knowledge distillation transfers "dark knowledge" from a large teacher model to a smaller student model, yielding a highly efficient network. To improve network's generalization ability, existing works use a larger temperature coefficient for knowledge distillation. Nevertheless, these methods may…