Patch-Level Knowledge Distillation and Regularization for Missing Modality Medical Image Segmentation

Abstract

In the context of medical image segmentation, complementary information among multi-modality images can improve segmentation performance. However, acquiring the complete multi-modality data in clinical settings is difficult. To tackle this problem, we propose a novel multi-modality knowledge distillation segmentation framework, which allows the inference performance of single-modality closer to that of multi-modality. In order to facilitate the extraction of valuable information from the multi-modality teacher network, we first introduce a subtask named patch-selection to distill the patch-level knowledge and improve the generalization capacity of networks simultaneously. Moreover, we employ contrastive learning distillation by defining patch-level positive and negative pairs in embedding, which can encourage the student network to extract more potential information from single-modality input and better understand the similarities and differences with the teacher network in representations. The evaluation process on the BraTS 2018 dataset shows the state-of-the-art performance of our method.