Guixu Zhang*

Harmonizing knowledge Transfer in Neural Network with Unified Distillation

"Knowledge distillation (KD), known for its ability to transfer knowledge from a cumbersome network (teacher) to a lightweight one (student) without altering the architecture, has been garnering increasing attention. Two primary categories emerge within KD methods: feature-based, focusing on interme…