2026
LumiNet: Perception-Driven Knowledge Distillation via Statistical Logit Calibration
MD ISMAIL HOSSAIN, M M Lutfe Elahi, Sameera Ramasinghe, Ali Cheraghian, Fuad Rahman, Nabeel Mohammed +1
ICML 2026poster
In the knowledge distillation literature, feature-based methods have dominated due to their ability to effectively tap into extensive teacher models. In contrast, logit-based approaches, which aim to distill `dark knowledge' from teachers, typically exhibit inferior performance compared to feature-b…