Distill To Detect: Amplifying Anomalies in Backdoor Models through Knowledge Distillation

Abstract

Backdoor attacks represent a significant threat to the security of deep learning models. Due to the stealthiness of backdoor attacks, effectively detecting whether a model has been compromised by such attacks remains a major challenge. Previous backdoor detection methods either rely on backdoor datasets to identify anomalies in backdoor samples or depend on prior knowledge of existing backdoor attacks. This results in difficulties in detecting backdoors when backdoor samples are unavailable and leads to poor generalization capabilities when addressing new attack methods. This work proposes a novel approach for detecting backdoor attacks called Distill To Detect (D2D), that does not depend on backdoor samples or any prior knowledge. It utilizes knowledge distillation to amplify more general and universal backdoor anomalies exhibited on clean samples for detection. This approach is not only more efficient but also exhibits strong generalization capabilities, enabling the detection of most backdoor attacks with low time and computational costs. We tested our approach against six backdoor attacks and three different model architectures, demonstrating the effectiveness of our proposed method.