M M Lutfe Elahi

LumiNet: Perception-Driven Knowledge Distillation via Statistical Logit Calibration

In the knowledge distillation literature, feature-based methods have dominated due to their ability to effectively tap into extensive teacher models. In contrast, logit-based approaches, which aim to distill `dark knowledge' from teachers, typically exhibit inferior performance compared to feature-b…