Masked Siamese Networks for Label-Efficient Learning
Mahmoud Assran, Mathilde Caron, Ishan Misra, Piotr Bojanowski, Florian Bordes, Pascal Vincent, Armand Joulin, Michael Rabbat
Abstract
"We propose Masked Siamese Networks (MSN), a self-supervised learning framework for learning image representations. Our approach matches the representation of an image view containing randomly masked patches to the representation of the original unmasked image. This self-supervised pre-training strategy is particularly scalable when applied to Vision Transformers since only the unmasked patches are processed by the network. As a result, MSNs improve the scalability of joint-embedding architectures, while producing representations of a high semantic level that perform competitively on low-shot image classification. For instance, on ImageNet-1K, with only 5,000 annotated images, our large MSN models achieves 72.1% top-1 accuracy, and with 1% of ImageNet-1K labels, we achieve 75.1% top-1 accuracy, setting a new state-of-the-art for self-supervised learning on this benchmark. Our code is publicly available at https://github.com/facebookresearch/msn."
BibTeX
@inproceedings{eccv2022_maskedsiamesenet,
title = {Masked Siamese Networks for Label-Efficient Learning},
author = {Mahmoud Assran and Mathilde Caron and Ishan Misra and Piotr Bojanowski and Florian Bordes and Pascal Vincent and Armand Joulin and Michael Rabbat and Nicolas Ballas},
booktitle = {ECCV 2022},
year = {2022}
}