Exploiting Foundation Models for Label-Efficient Few-Shot Learning via Feature Coupling: A Case Study of cardiac CT Segmentation

Abstract

The scarcity of labeled data poses a significant challenge for deep learning-based medical image segmentation. To address this, this study introduces the novel Foundation Model-based Few-Shot Segmentation (FM-FSS) paradigm. FM-FSS capitalizes on the knowledge distilled from pre-trained foundation models, such as the Segment Anything Model, to enhance segmentation performance in few-shot scenarios. The paradigm designs a feature coupling module that synergizes SAM’s powerful feature extraction capabilities with nnU-Net’s self-configuration strategy, enabling accurate segmentation with minimal labeled data and optional manual prompt inputs. Extensive experiments on a publicly available cardiac CT dataset demonstrate that FM-FSS outperforms state-of-the-art segmentation models. With only 20 labeled images, our method achieves an average Dice score of 94.33% and an ASD of 1.10 mm. Moreover, FM-FSS maintains its label-efficient performance in a one-shot setup, reducing the annotation requirements by at least fourfold. The code and pre-trained models will be released upon acceptance.