SSDViT: Exploring Siamese and Self Distillation in ViTs for Generalizable Person Re-identification

Abstract

Person re-identification (re-ID) models often fail to generalize well when deployed to unseen camera networks with domain shift. Domain generalization (DG) aims to address this dilemma by training a model on source domains that learns domain-invariant, and hence generalizable representations. Most methods in the literature are based on Convolutional Neural Networks (CNNs), while the DG performance of Vision Transformers (ViTs) remains relatively unexplored. In contrast to CNNs, ViTs lack explicit inductive biases, which makes it extremely data-hungry and easily overfit to source domains. In this paper, we investigate the generalization ability of ViTs and propose a novel Siamese and Self Distillation Vision Transformer (SSDViT) framework towards addressing the DG re-ID problem. To be specific, Siamese Distillation exploits the weak-to-strong consistency regularization at the image level to enforce the strongly perturbed image to yield consistent prediction with its weakly perturbed version, which provides a simple yet effective solution to introduce invariance bias. On the other hand, Self Distillation seeks to impose consistency constraints at the feature level by leveraging intermediate knowledge to improve the robustness of learned representations. The proposed unified framework pursues the equivalence of predictions at both the image and embedding levels, which underpins the generalization capabilities of learned representations and alleviates the risk of overfitting to source domains. Without bells and whistles, the proposed approach achieves a new state-of-the-art on various DG re-ID benchmarks. Codes are available at https://github.com/yJCTrans/SSDViT.