Qingle Huang

Exploring Inter-Channel Correlation for Diversity-Preserved Knowledge Distillation

Knowledge Distillation has shown very promising ability in transferring learned representation from the larger model (teacher) to the smaller one (student). Despite many efforts, prior methods ignore the important role of retaining inter-channel correlation of features, leading to the lack of captur…