A Weakly Supervised Semantic Segmentation Model with Enhanced CLIP Feature Extraction

Abstract

This paper addresses the limitations of the Contrastive Language-Image Pre-training (CLIP) model’s image encoder and proposes a segmentation model WSSS-ECFE with enhanced CLIP feature extraction, aiming to improve the performance of the Weakly Supervised Semantic Segmentation (WSSS) task. WSSS-ECFE employs the Enhanced Bottleneck module proposed in this paper and adds dynamic residual connection to improve the model’s processing effect on complex scenes. In terms of implementation, the Enhanced Bottleneck module employs the Swish activation function and the Depthwise Separable Convolution to enhance the feature extraction and segmentation capability of the model, and uses multiple attention mechanisms to further optimize the feature representation and segmentation accuracy. The WSSS task on the public datasets PASCAL VOC 2012 and MS COCO 2014 achieves 82.6% and 56.3% mean intersection over union (mIoU), achieving state-of-the-art performance in models with low resource requirements.