BrainSegNet: A Novel Framework for Whole-Brain MRI Parcellation Enhanced by Large Models

TL;DR

BrainSegNet tackles the challenge of accurate whole-brain MRI parcellation into 95 regions. It adapts SAM by integrating a SAM-based U-Net encoder, a multi-scale attention decoder (ASPP) with Channel and Spatial Attention (CSA), and a Boundary Refinement module, trained with a hybrid loss $L_{seg}=\alpha L_{ce}+\beta L_{Dice}$ where $\alpha=0.2$, $\beta=0.8$. On the HCP dataset, BrainSegNet outperforms state-of-the-art methods, demonstrating improved accuracy particularly on small or irregular regions, and ablation confirms the contribution of each component. The approach offers a scalable, high-precision solution for automated whole-brain parcellation with potential impact on clinical planning and large-scale neuroimaging studies.

Abstract

Whole-brain parcellation from MRI is a critical yet challenging task due to the complexity of subdividing the brain into numerous small, irregular shaped regions. Traditionally, template-registration methods were used, but recent advances have shifted to deep learning for faster workflows. While large models like the Segment Anything Model (SAM) offer transferable feature representations, they are not tailored for the high precision required in brain parcellation. To address this, we propose BrainSegNet, a novel framework that adapts SAM for accurate whole-brain parcellation into 95 regions. We enhance SAM by integrating U-Net skip connections and specialized modules into its encoder and decoder, enabling fine-grained anatomical precision. Key components include a hybrid encoder combining U-Net skip connections with SAM's transformer blocks, a multi-scale attention decoder with pyramid pooling for varying-sized structures, and a boundary refinement module to sharpen edges. Experimental results on the Human Connectome Project (HCP) dataset demonstrate that BrainSegNet outperforms several state-of-the-art methods, achieving higher accuracy and robustness in complex, multi-label parcellation.

Figures (2)

• Figure 1: Overview of the proposed BrainSegNet framework for whole-brain multi-label parcellation.
• Figure 2: Visualization of a representative sagittal slice: GT (ground truth), FastSurfer, MASAM, SAM, and BrainSegNet. The regions circled in red indicate statistically significant differences.