Brain MRI Segmentation with Language-Driven Detection and Context-Aware Descriptions

Abstract

Brain MRI segmentation is critical for diagnosis and treatment planning, but existing methods are often limited by their task-specific designs and lack of generalizability. A significant challenge lies in integrating multiple brain imaging datasets with varying structural labels for comprehensive analysis. To address this, we propose GroundingSeg, a language-driven framework that unifies segmentation and object detection with phrase grounding to enhance segmentation outcomes. GroundingSeg leverages semantic-aware detection queries and auto-generated descriptions to provide fine-grained and context-aware segmentation of brain structures, using object detection to improve segmentation precision. Our results show that GroundingSeg outperforms existing methods in adaptability and precision, offering a robust solution for brain MRI segmentation. Our source code is available at https://github.com/ABC67876/GroundingSeg/.