Mingxuan Song

Adversarial Distillation Based on Slack Matching and Attribution Region Alignment

Adversarial distillation (AD) is a highly effective method for enhancing the robustness of small models. Contrary to expectations a high-performing teacher model does not always result in a more robust student model. This is due to two main reasons. First when there are significant differences in pr…