Hengyu Luo

Knowledge Distillation as Decontamination? Revisiting the “Data Laundering” Concern

Concerns have been raised that knowledge distillation may transfer test-set knowledge from a contaminated teacher to a clean student—a “data laundering” effect that potentially threatens evaluation integrity. In this paper, we assess the severity of this phenomenon. If these concerns regarding data…