Symmetric Bi-branch Modality-search Aggregation Network for Multi-modal Liver Segmentation

Abstract

Medical image segmentation is crucial for diagnosis and surgical planning of liver diseases. The existing methods mainly focus on global or local features and neglect spatial dependencies among modalities and blurred boundaries. To tackle these challenges, we propose a symmetric bi-branch modality-search aggregation network (SBMANet). Specifically, we first design a symmetric network with dual encoder-decoder structure. Each encoder fuses two adjacent modal features to improve the intra-modal spatial information while the decoder can achieve accurate localization of segmented targets. To fully exploit multi-modal inter-modal dependencies, a hybrid Hadamard multimodal fusion module (HHMF) is proposed. Finally, we establish an adaptive modality-channel-search module by incorporating bi-branch features to automatically compute weights for each channel in different modalities. Extensive experiments on DLDS demonstrate that the proposed network outperforms existing state-of-the-art 3D segmentation networks. The code is available at the website: https://github.com/fangchj2002/SBMANet.