Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Scaling Ultrasound Volumetric Reconstruction via Mobile Augmented Reality

Kian Wei Ng, Yujia Gao, Deborah Khoo, Ying Zhen Tan, Chengzheng Mao, Haojie Cheng, Andrew Makmur, Kee Yuan Ngiam, Serene Goh, Eng Tat Khoo

TL;DR

The paper tackles the challenge of scalable, accurate volumetric ultrasound for nodules by introducing MARVUS, a Mobile Augmented Reality Ultrasound system that converts 2D-US into reliable 3D reconstructions using a low-cost workflow. It combines a simple single-frame intrinsic calibration phantom, ArUco-based extrinsics, EdgeTAM segmentation, voxelization, and marching cubes, all visualized through AR overlays to guide acquisition and verification. In a phantom study with experienced clinicians, MARVUS significantly improved volumetric accuracy (mean differences down to $0.161$–$0.270\ \mathrm{cm^3}$) and reduced inter-user variability (≈$0.417\ \mathrm{cm^3}$), with AR visuals boosting perceived usability and confidence. The approach is designed to be accessible and scalable, potentially enhancing US-based screening, diagnosis, and treatment planning while reducing hardware costs. Follow-up work will explore broader clinical scenarios, automated biopsy guidance, and AR-assisted training, leveraging the low-cost, cross-specialty framework.

Abstract

Accurate volumetric characterization of lesions is essential for oncologic diagnosis, risk stratification, and treatment planning. While imaging modalities such as Computed Tomography provide high-quality 3D data, 2D ultrasound (2D-US) remains the preferred first-line modality for breast and thyroid imaging due to cost, portability, and safety factors. However, volume estimates derived from 2D-US suffer from high inter-user variability even among experienced clinicians. Existing 3D ultrasound (3D-US) solutions use specialized probes or external tracking hardware, but such configurations increase costs and diminish portability, constraining widespread clinical use. To address these limitations, we present Mobile Augmented Reality Volumetric Ultrasound (MARVUS), a resource-efficient system designed to increase accessibility to accurate and reproducible volumetric assessment. MARVUS is interoperable with conventional ultrasound (US) systems, using a foundation model to enhance cross-specialty generalization while minimizing hardware requirements relative to current 3D-US solutions. In a user study involving experienced clinicians performing measurements on breast phantoms, MARVUS yielded a substantial improvement in volume estimation accuracy (mean difference: 0.469 cm3) with reduced inter-user variability (mean difference: 0.417 cm3). Additionally, we prove that augmented reality (AR) visualizations enhance objective performance metrics and clinician-reported usability. Collectively, our findings suggests that MARVUS can enhance US-based cancer screening, diagnostic workflows, and treatment planning in a scalable, cost-conscious, and resource-efficient manner. Usage video demonstration available (https://youtu.be/m4llYcZpqmM).

Scaling Ultrasound Volumetric Reconstruction via Mobile Augmented Reality

TL;DR

The paper tackles the challenge of scalable, accurate volumetric ultrasound for nodules by introducing MARVUS, a Mobile Augmented Reality Ultrasound system that converts 2D-US into reliable 3D reconstructions using a low-cost workflow. It combines a simple single-frame intrinsic calibration phantom, ArUco-based extrinsics, EdgeTAM segmentation, voxelization, and marching cubes, all visualized through AR overlays to guide acquisition and verification. In a phantom study with experienced clinicians, MARVUS significantly improved volumetric accuracy (mean differences down to –) and reduced inter-user variability (≈), with AR visuals boosting perceived usability and confidence. The approach is designed to be accessible and scalable, potentially enhancing US-based screening, diagnosis, and treatment planning while reducing hardware costs. Follow-up work will explore broader clinical scenarios, automated biopsy guidance, and AR-assisted training, leveraging the low-cost, cross-specialty framework.

Abstract

Accurate volumetric characterization of lesions is essential for oncologic diagnosis, risk stratification, and treatment planning. While imaging modalities such as Computed Tomography provide high-quality 3D data, 2D ultrasound (2D-US) remains the preferred first-line modality for breast and thyroid imaging due to cost, portability, and safety factors. However, volume estimates derived from 2D-US suffer from high inter-user variability even among experienced clinicians. Existing 3D ultrasound (3D-US) solutions use specialized probes or external tracking hardware, but such configurations increase costs and diminish portability, constraining widespread clinical use. To address these limitations, we present Mobile Augmented Reality Volumetric Ultrasound (MARVUS), a resource-efficient system designed to increase accessibility to accurate and reproducible volumetric assessment. MARVUS is interoperable with conventional ultrasound (US) systems, using a foundation model to enhance cross-specialty generalization while minimizing hardware requirements relative to current 3D-US solutions. In a user study involving experienced clinicians performing measurements on breast phantoms, MARVUS yielded a substantial improvement in volume estimation accuracy (mean difference: 0.469 cm3) with reduced inter-user variability (mean difference: 0.417 cm3). Additionally, we prove that augmented reality (AR) visualizations enhance objective performance metrics and clinician-reported usability. Collectively, our findings suggests that MARVUS can enhance US-based cancer screening, diagnostic workflows, and treatment planning in a scalable, cost-conscious, and resource-efficient manner. Usage video demonstration available (https://youtu.be/m4llYcZpqmM).
Paper Structure (12 sections, 3 equations, 4 figures, 3 tables)

This paper contains 12 sections, 3 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Ultrasound Calibration (Intrinsics)
  4. Ultrasound Calibration (Extrinsics)
  5. Nodule Reconstruction
  6. Nodule Assessment (Augmented Reality)
  7. User Study Design
  8. Results
  9. Ultrasound Calibration
  10. User Study (Volumetry)
  11. User Study (Survey Response)
  12. Conclusion

Figures (4)

  • Figure 1: The MARVUS workflow. Calibration -- US probe with strapped ArUco markers (a) is tracked via a mobile phone (b). Calibration is done via a simple phantom (c), with region-of-interest and keypoints detected from a raw screen capture (d). Reconstruction -- A freehand sweep imaging a nodule is done with accompanying augmented reality visualizations (e). Nodule segmentation is obtained in a semi-automated manner (f) and compounded into 3D for volume estimation. Verification -- The 3D mesh is verified in augmented reality (g) by comparing the mesh-plane intersection with a live ultrasound feed (green denotes intersection between the mesh and the live US image).
  • Figure 2: US Calibration Phantom. A single-piece 3D-printed phantom is used for single-frame spatial calibration using US-visible "ledges" (a) with intersections enabling calibration. The "ledges" $(L_{1},L_{2},L_{3})$ are brightest when viewed directly in-plane due to US physics. (b) Visible markers are used for extrinsics calibration (right).
  • Figure 3: Nodule Reconstruction Steps. (a) Segmentation of tracked US sequence produces a point cloud. (b) Voxelization resamples and fills sparsely sampled locations. (c) A meshing algorithm is applied to smoothen noise-induced highly irregular surfaces.
  • Figure 4: Nodule Volume Estimation. Our AR 3D-US setup improves accuracy and reduces operator variability (left), notably for complex non-spherical nodules (right).