2022
Boosting Contrastive Learning with Relation Knowledge Distillation
AAAI 2022technical
While self-supervised representation learning (SSL) has proved to be effective in the large model, there is still a huge gap between the SSL and supervised method in the lightweight model when following the same solution. We delve into this problem and find that the lightweight model is prone to col…