Atlas-Assisted Segment Anything Model for Fetal Brain MRI (FeTal-SAM)

TL;DR

This work tackles two key issues in fetal brain MRI segmentation: the need for flexible, on-demand labeling definitions and the interpretability of segmentation decisions with respect to image contrast versus spatial priors. It introduces FeTal-SAM, an atlas-assisted adaptation of the Segment Anything Model (SAM) that uses multi-atlas to subject registration, dense atlas prompts from label templates, and a bounding-box prompt to drive the SAM decoder, followed by 3D reconstruction via STAPLE fusion. Evaluations on the dHCP and CRL datasets show that FeTal-SAM achieves Dice scores comparable to dataset-specific 3D baselines for well-contrasted structures (e.g., cortical plate, cerebellum) while enabling flexible, user-defined parcellations without retraining; smaller, low-contrast structures remain challenging due to inherent MRI contrast and 2D prompt limitations. By unifying atlas priors with a foundation model, FeTal-SAM offers a general-purpose, adaptable fetal brain segmentation tool and lays groundwork for improved 3D spatial priors and uncertainty quantification in clinical workflows.

Abstract

This paper presents FeTal-SAM, a novel adaptation of the Segment Anything Model (SAM) tailored for fetal brain MRI segmentation. Traditional deep learning methods often require large annotated datasets for a fixed set of labels, making them inflexible when clinical or research needs change. By integrating atlas-based prompts and foundation-model principles, FeTal-SAM addresses two key limitations in fetal brain MRI segmentation: (1) the need to retrain models for varying label definitions, and (2) the lack of insight into whether segmentations are driven by genuine image contrast or by learned spatial priors. We leverage multi-atlas registration to generate spatially aligned label templates that serve as dense prompts, alongside a bounding-box prompt, for SAM's segmentation decoder. This strategy enables binary segmentation on a per-structure basis, which is subsequently fused to reconstruct the full 3D segmentation volumes. Evaluations on two datasets, the dHCP dataset and an in-house dataset demonstrate FeTal-SAM's robust performance across gestational ages. Notably, it achieves Dice scores comparable to state-of-the-art baselines which were trained for each dataset and label definition for well-contrasted structures like cortical plate and cerebellum, while maintaining the flexibility to segment any user-specified anatomy. Although slightly lower accuracy is observed for subtle, low-contrast structures (e.g., hippocampus, amygdala), our results highlight FeTal-SAM's potential to serve as a general-purpose segmentation model without exhaustive retraining. This method thus constitutes a promising step toward clinically adaptable fetal brain MRI analysis tools.

Figures (6)

• Figure 1: The Proposed Multi-Assisted Driven Fetal Brain SAM Segmentation Framework
• Figure 2: Examples of under and over-prompting issues when a registered image atlas was used for target tissue initial localization
• Figure 3: FeTal-SAM 3D segmentation results comparison to nnUnet and SwinUNetR
• Figure 4: Graphical segmentation results from DHCP test cases: A sample case with GA at 22 weeks is shown on the top three rows, and a sample case with GA at 30 weeks is shown at the bottom, respectively.
• Figure 5: Graphical Comparison of segmentation errors for different tissue structures with a test sample from our CRL dataset. The left panel shows the results from FeTal-SAM and nnUNet for cortical plate, sub-plate and cerebellum. The right panel shows results of more challenging small tissue structures, including hippocampus and amygdala. The ground truth (GT), over segmentation and under segmentation are displayed in red, green and blue, respectively.