FetusMap: Fetal Pose Estimation in 3D Ultrasound

TL;DR

This paper proposes to estimate the 3D pose of fetus in US volumes to facilitate its quantitative analyses in global and local scales and proposes a self-supervised learning (SSL) framework to finetune the deep network to form visually plausible pose predictions.

Abstract

The 3D ultrasound (US) entrance inspires a multitude of automated prenatal examinations. However, studies about the structuralized description of the whole fetus in 3D US are still rare. In this paper, we propose to estimate the 3D pose of fetus in US volumes to facilitate its quantitative analyses in global and local scales. Given the great challenges in 3D US, including the high volume dimension, poor image quality, symmetric ambiguity in anatomical structures and large variations of fetal pose, our contribution is three-fold. (i) This is the first work about 3D pose estimation of fetus in the literature. We aim to extract the skeleton of whole fetus and assign different segments/joints with correct torso/limb labels. (ii) We propose a self-supervised learning (SSL) framework to finetune the deep network to form visually plausible pose predictions. Specifically, we leverage the landmark-based registration to effectively encode case-adaptive anatomical priors and generate evolving label proxy for supervision. (iii) To enable our 3D network perceive better contextual cues with higher resolution input under limited computing resource, we further adopt the gradient check-pointing (GCP) strategy to save GPU memory and improve the prediction. Extensively validated on a large 3D US dataset, our method tackles varying fetal poses and achieves promising results. 3D pose estimation of fetus has potentials in serving as a map to provide navigation for many advanced studies.

Figures (6)

• Figure 1: 3D pose estimation of fetus in US volumes. (a) A sectional view of a fetus in US volume. (b) An instance of 3D fetal pose with 16 landmark indexes and 15 colored segments. (c) All the pose annotations of 152 fetuses in our dataset. Large variations exist when referring to (b). Better view in color version.
• Figure 2: Schematic view of our proposed framework for on-line refinement.
• Figure 3: Our proposed U-net like architecture for landmark detection.
• Figure 4: Illustration of the forward pass and gradient re-computation in backward pass of the GCP. Dotted circle denotes the node in the computation graph to be emptied.
• Figure 5: PCK curves for 3 fetal landmarks. x axis is the distance threshold. SSLGCP (dotted green curve) gets the best results among all the competitors.