David Gonzalo Chaves Campos

Weighted Mutual Learning with Diversity-Driven Model Compression

Online distillation attracts attention from the community as it simplifies the traditional two-stage knowledge distillation process into a single stage. Online distillation collaboratively trains a group of peer models, which are treated as students, and all students gain extra knowledge from each o…