Fetal-BET: Brain Extraction Tool for Fetal MRI

TL;DR

This work developed and validated deep learning methods, by exploiting the power of the U-Net style architectures, the attention mechanism, feature learning across multiple MRI modalities, and data augmentation for fast, accurate, and generalizable automatic fetal brain extraction.

Abstract

Fetal brain extraction is a necessary first step in most computational fetal brain MRI pipelines. However, it has been a very challenging task due to non-standard fetal head pose, fetal movements during examination, and vastly heterogeneous appearance of the developing fetal brain and the neighboring fetal and maternal anatomy across various sequences and scanning conditions. Development of a machine learning method to effectively address this task requires a large and rich labeled dataset that has not been previously available. As a result, there is currently no method for accurate fetal brain extraction on various fetal MRI sequences. In this work, we first built a large annotated dataset of approximately 72,000 2D fetal brain MRI images. Our dataset covers the three common MRI sequences including T2-weighted, diffusion-weighted, and functional MRI acquired with different scanners. Moreover, it includes normal and pathological brains. Using this dataset, we developed and validated deep learning methods, by exploiting the power of the U-Net style architectures, the attention mechanism, multi-contrast feature learning, and data augmentation for fast, accurate, and generalizable automatic fetal brain extraction. Our approach leverages the rich information from multi-contrast (multi-sequence) fetal MRI data, enabling precise delineation of the fetal brain structures. Evaluations on independent test data show that our method achieves accurate brain extraction on heterogeneous test data acquired with different scanners, on pathological brains, and at various gestational stages. This robustness underscores the potential utility of our deep learning model for fetal brain imaging and image analysis.

Figures (6)

• Figure 1: Examples of multi-modal in-utero MRI images including T2-weighted, diffusion-weighted, and functional MRI. The first row shows in-plane views and the second and the third rows show out-of-plane views. These examples highlight some of the factors that make fetal brain extraction especially challenging such as motion artifacts, anisotropic resolution, heterogeneous contrast, an the highly variable shape and features of the anatomy based on the gestational age and the position of the fetus.
• Figure 2: Architecture of the U-Net with Attention Gates (AG) known as Attention U-Net. The backbone U-Net architecture can be achieved if the AG units are ignored. In the Attention U-Net, AGs filter the features that are propagated through the skip connections by using the contextual information of features extracted in coarser scales. This is achieved by adding the decoder output of a coarser scale to the output of every skip connection from the encoder after $1 \times 1$ convolutions. The output then passes through ReLU and sigmoid activation functions and is multiplied to the coarser level decoder input.
• Figure 3: Boxplots of Dice similarity coeffiicent (DSC) and Intersection-over-Union (IoU) for different sequences (T2-weighted (T2W), diffusion-weighted (DWI), and functional MRI (fMRI)), T2W data characteristics (Typical, Abnormalities, Artifacts, and Twins pregnancies), and model architectures (U-Net, Dynamic U-Net, and Attention U-Net). Higher DSC and IoU values indicate greater segmentation accuracy. The U-Net and Attention U-Net models achieved higher median Dice scores overall compared to the Dynamic U-Net model. The asterisks displayed on the top left plot serve as visual indicators of the statistical significance associated with the differences observed between the groups using a paired t-test. The asterisks displayed on the first plot serve as visual indicators of the statistical significance associated with the differences observed between the groups. Significance levels: (ns) $p > 0.05$, (*) $0.01 < p \leq 0.05$, (**) $0.001 < p \leq 0.01$, (***) $0.0001 < p \leq 0.001$, (****) $p \leq 0.0001$.
• Figure 4: Boxplots of Dice similarity coeffiicent (DSC) and Intersection-over-Union (IoU) for different MRI sequences (T2-weighted (T2W), diffusion-weighted (DWI), and functional MRI (fMRI)), comparing the extraction performance of different Attention U-Net model architectures ((Single, Aug): trained on a single sequence (corresponding) with augmentation, (All, No Aug): trained on all sequences with no augmentation, and (All, Aug): trained on all sequences with augmentation). Left plots show the results on our test data and right plots show the results on our out-of-distribution test data. The best result is achieved by Attention U-Net trained on all sequences with augmentation. This indicates that leveraging multiple complementary sequences and expanded training data through augmentation can enhance deep learning model performance.
• Figure 5: Representative examples of segmentations produced by different models (U-Net, DynU-Net, AttU-Net) on T2w slices. On each image, the blue curve shows the outline of the reference brain mask drawn manually by an experienced annotator, while the red curve shows the contour of the segmentation mask predicted by the deep learning method.