Semantic Information in Contrastive Learning
Shengjiang Quan, Masahiro Hirano, Yuji Yamakawa
Abstract
This work investigates the functionality of Semantic information in Contrastive Learning (SemCL). An advanced pretext task is designed: a contrast is performed between each object and its environment, taken from a scene. This allows the SemCL pretrained model to extract objects from their environment in an image, significantly improving the spatial understanding of the pretrained models. Downstream tasks of semantic/instance segmentation, object detection and depth estimation are implemented on PASCAl VOC, Cityscapes, COCO, KITTI, etc. SemCL pretrained models substantially outperform ImageNet pretrained counterparts and are competitive with well-known works on downstream tasks. The results suggest that a dedicated pretext task leveraging semantic information can be powerful in benchmarks related to spatial understanding. The code is available at https://github.com/sjiang95/semcl.
BibTeX
@inproceedings{iccv2023_semanticinformat,
title = {Semantic Information in Contrastive Learning},
author = {Shengjiang Quan and Masahiro Hirano and Yuji Yamakawa},
booktitle = {ICCV 2023},
year = {2023}
}