Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Shape Completion in the Dark: Completing Vertebrae Morphology from 3D Ultrasound

Miruna-Alexandra Gafencu, Yordanka Velikova, Mahdi Saleh, Tamas Ungi, Nassir Navab, Thomas Wendler, Mohammad Farid Azampour

TL;DR

This paper tackles the challenge of interpreting ultrasound-based spine data by developing a 3D vertebrae shape completion framework trained on physics-informed synthetic data. The authors introduce a two-network variational pipeline (PMNet and RENet) that completes occluded vertebrae from partial US-like views, with a Poisson surface reconstruction step to generate meshes. They demonstrate consistent performance on synthetic and real US/CT data, preserve key anatomical landmarks such as the spinous process and facet joints, and show that incorporating US physics into data generation significantly improves accuracy. The approach holds promise for ultrasound-guided spine interventions by providing complete 3D visualization without additional imaging modalities, and code/data will be publicly released.

Abstract

Purpose: Ultrasound (US) imaging, while advantageous for its radiation-free nature, is challenging to interpret due to only partially visible organs and a lack of complete 3D information. While performing US-based diagnosis or investigation, medical professionals therefore create a mental map of the 3D anatomy. In this work, we aim to replicate this process and enhance the visual representation of anatomical structures. Methods: We introduce a point-cloud-based probabilistic DL method to complete occluded anatomical structures through 3D shape completion and choose US-based spine examinations as our application. To enable training, we generate synthetic 3D representations of partially occluded spinal views by mimicking US physics and accounting for inherent artifacts. Results: The proposed model performs consistently on synthetic and patient data, with mean and median differences of 2.02 and 0.03 in CD, respectively. Our ablation study demonstrates the importance of US physics-based data generation, reflected in the large mean and median difference of 11.8 CD and 9.55 CD, respectively. Additionally, we demonstrate that anatomic landmarks, such as the spinous process (with reconstruction CD of 4.73) and the facet joints (mean distance to GT of 4.96mm) are preserved in the 3D completion. Conclusion: Our work establishes the feasibility of 3D shape completion for lumbar vertebrae, ensuring the preservation of level-wise characteristics and successful generalization from synthetic to real data. The incorporation of US physics contributes to more accurate patient data completions. Notably, our method preserves essential anatomic landmarks and reconstructs crucial injections sites at their correct locations. The generated data and source code will be made publicly available (https://github.com/miruna20/Shape-Completion-in-the-Dark).

Shape Completion in the Dark: Completing Vertebrae Morphology from 3D Ultrasound

TL;DR

This paper tackles the challenge of interpreting ultrasound-based spine data by developing a 3D vertebrae shape completion framework trained on physics-informed synthetic data. The authors introduce a two-network variational pipeline (PMNet and RENet) that completes occluded vertebrae from partial US-like views, with a Poisson surface reconstruction step to generate meshes. They demonstrate consistent performance on synthetic and real US/CT data, preserve key anatomical landmarks such as the spinous process and facet joints, and show that incorporating US physics into data generation significantly improves accuracy. The approach holds promise for ultrasound-guided spine interventions by providing complete 3D visualization without additional imaging modalities, and code/data will be publicly released.

Abstract

Purpose: Ultrasound (US) imaging, while advantageous for its radiation-free nature, is challenging to interpret due to only partially visible organs and a lack of complete 3D information. While performing US-based diagnosis or investigation, medical professionals therefore create a mental map of the 3D anatomy. In this work, we aim to replicate this process and enhance the visual representation of anatomical structures. Methods: We introduce a point-cloud-based probabilistic DL method to complete occluded anatomical structures through 3D shape completion and choose US-based spine examinations as our application. To enable training, we generate synthetic 3D representations of partially occluded spinal views by mimicking US physics and accounting for inherent artifacts. Results: The proposed model performs consistently on synthetic and patient data, with mean and median differences of 2.02 and 0.03 in CD, respectively. Our ablation study demonstrates the importance of US physics-based data generation, reflected in the large mean and median difference of 11.8 CD and 9.55 CD, respectively. Additionally, we demonstrate that anatomic landmarks, such as the spinous process (with reconstruction CD of 4.73) and the facet joints (mean distance to GT of 4.96mm) are preserved in the 3D completion. Conclusion: Our work establishes the feasibility of 3D shape completion for lumbar vertebrae, ensuring the preservation of level-wise characteristics and successful generalization from synthetic to real data. The incorporation of US physics contributes to more accurate patient data completions. Notably, our method preserves essential anatomic landmarks and reconstructs crucial injections sites at their correct locations. The generated data and source code will be made publicly available (https://github.com/miruna20/Shape-Completion-in-the-Dark).
Paper Structure (31 sections, 15 figures, 2 tables, 2 algorithms)

This paper contains 31 sections, 15 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Materials and Methods
  3. Synthetic Data Generation
  4. Accounting for multiple spine curvatures
  5. Generation of -consistent partial views of the spine
  6. Angle of incidence-aware ray-casting
  7. Account for Ultrasound Scattering
  8. Masking spine into separate vertebrae views
  9. Vertebrae Shape Completion
  10. Datasets description
  11. Shape completion metrics
  12. Experiments
  13. Experimental Setup of Our Method
  14. Anatomical Landmarks Preservation
  15. Ablation Study
  16. ...and 16 more sections

Figures (15)

  • Figure 1: Overview of the training pipeline of our proposed method. First data generation is performed, followed by shape completion and post processing.
  • Figure 2: US scan of L1, L2 and L3 vertebrae levels of a spine phantom. These images exemplify the partial view of the vertebral arch as well as US-specific artifacts. We can see the effects of acoustic shadowing in the partially visible spinous process, highlighted with a bounding box on the left image.
  • Figure 3: Performance comparison (in terms of Chamfer Distance (CD), Earth's Mover's Distance (EMD) and F1-Score) of our full pipeline with two different shape completion approaches (VRCNet (blue) and PCN (orange)) on synthetic and patient data, as well as results of the ablation studies.
  • Figure 4: Patient data results obtained with the full pipeline comparing two shape completion networks as well as two ablation studies. Given our partial input (red), We compare the reconstruction (blue) with the ground truth (green) and report three metrics: CD, EMD, and F1. We visualized the input and each completed shape PC from two views along the frontal and longitudinal axes.
  • Figure 5: Comparison of spine mesh ray-casting when (a) the angle of incidence is not considered (b) the angle of incidence is considered. The resulting point cloud contains more shadows and is, therefore, more similar to the US view of the spine.
  • ...and 10 more figures