Automatic 3D Multi-modal Ultrasound Segmentation of Human Placenta using Fusion Strategies and Deep Learning

TL;DR

An automatic three-dimensional multi-modal ultrasound segmentation of the human placenta using deep learning combined with different fusion strategies is proposed and is effective in segmenting the placenta from 3D ultrasound scans in a fully automated manner and is robust to quality variation of the datasets.

Abstract

Purpose: Ultrasound is the most commonly used medical imaging modality for diagnosis and screening in clinical practice. Due to its safety profile, noninvasive nature and portability, ultrasound is the primary imaging modality for fetal assessment in pregnancy. Current ultrasound processing methods are either manual or semi-automatic and are therefore laborious, time-consuming and prone to errors, and automation would go a long way in addressing these challenges. Automated identification of placental changes at earlier gestation could facilitate potential therapies for conditions such as fetal growth restriction and pre-eclampsia that are currently detected only at late gestational age, potentially preventing perinatal morbidity and mortality. Methods: We propose an automatic three-dimensional multi-modal (B-mode and power Doppler) ultrasound segmentation of the human placenta using deep learning combined with different fusion strategies.We collected data containing Bmode and power Doppler ultrasound scans for 400 studies. Results: We evaluated different fusion strategies and state-of-the-art image segmentation networks for placenta segmentation based on standard overlap- and boundary-based metrics. We found that multimodal information in the form of B-mode and power Doppler scans outperform any single modality. Furthermore, we found that B-mode and power Doppler input scans fused at the data level provide the best results with a mean Dice Similarity Coefficient (DSC) of 0.849. Conclusion: We conclude that the multi-modal approach of combining B-mode and power Doppler scans is effective in segmenting the placenta from 3D ultrasound scans in a fully automated manner and is robust to quality variation of the datasets.

Figures (8)

• Figure 1: Visualisation of human placenta in three-dimensional ultrasound. Red contour shows placenta in axial, coronal, and sagittal views.
• Figure 2: Sample dataset showing B-mode ultrasound (top row), Power Doppler (PD) ultrasound (middle row), and Ground Truth (GT) mask (bottom row) for the axial, coronal, and saggital views.
• Figure 3: Sample dataset showing B-mode ultrasound (top row), Power Doppler (PD) ultrasound (middle row), and Ground Truth (GT) mask (bottom row) for the axial, coronal, and saggital views.
• Figure 4: Block diagram of the proposed methodology. (a) Training data in the form of three-dimensional B-mode and Power Doppler scans and annotated masks for network training. Fusion strategies such as early fusion, multi-stage fusion, and late fusion are applied combining two modalities. (b) Different medical image segmentation networks such as U-Net, U-Net++, and their variants. (c). Model inference for the three-dimensional B-mode scan predicting mask for the axial, coronal, and the sagittal views.
• Figure 5: Model architecture for different fusion strategies. Early fusion (left) concatenates original features at the input level. Multi-stage or joint fusion (middle) concatenate extracted features. Late fusion (right) aggregates predictions at the decision level. Neural network in the diagram denotes image segmentation network.