Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Automated Initial Probe Placement in Transthoracic Teleultrasound Using Human Mesh and Skeleton Recovery

Yu Chung Lee, David G. Black, Ryan S. Yeung, Septimiu E. Salcudean

Abstract

Cardiac and lung ultrasound are technically demanding because operators must identify patient-specific intercostal acoustic windows and then navigate between standard views by adjusting probe position, rotation, and force across different imaging planes. These challenges are amplified in teleultrasound when a novice or robot faces the difficult task of first placing the probe on the patient without in-person expert assistance. We present a framework for automating Patient registration and anatomy-informed Initial Probe placement Guidance (PIPG) using only RGB images from a calibrated camera. The novice first captures the patient using the camera on a mixed reality (MR) head-mounted display (HMD). An edge server then infers a patient-specific body-surface and skeleton model, with spatial smoothing across multiple views. Using bony landmarks from the predicted skeleton, we estimate the intercostal region and project the guidance back onto the reconstructed body surface. To validate the framework, we overlaid the reconstructed body mesh and the virtual probe pose guidance across multiple transthoracic echocardiography scan planes in situ and measured the quantitative placement error. Pilot experiments with healthy volunteers suggest that the proposed probe placement prediction and MR guidance yield consistent initial placement within anatomical variability acceptable for teleultrasound setup

Towards Automated Initial Probe Placement in Transthoracic Teleultrasound Using Human Mesh and Skeleton Recovery

Abstract

Cardiac and lung ultrasound are technically demanding because operators must identify patient-specific intercostal acoustic windows and then navigate between standard views by adjusting probe position, rotation, and force across different imaging planes. These challenges are amplified in teleultrasound when a novice or robot faces the difficult task of first placing the probe on the patient without in-person expert assistance. We present a framework for automating Patient registration and anatomy-informed Initial Probe placement Guidance (PIPG) using only RGB images from a calibrated camera. The novice first captures the patient using the camera on a mixed reality (MR) head-mounted display (HMD). An edge server then infers a patient-specific body-surface and skeleton model, with spatial smoothing across multiple views. Using bony landmarks from the predicted skeleton, we estimate the intercostal region and project the guidance back onto the reconstructed body surface. To validate the framework, we overlaid the reconstructed body mesh and the virtual probe pose guidance across multiple transthoracic echocardiography scan planes in situ and measured the quantitative placement error. Pilot experiments with healthy volunteers suggest that the proposed probe placement prediction and MR guidance yield consistent initial placement within anatomical variability acceptable for teleultrasound setup
Paper Structure (11 sections, 4 figures, 1 table)

This paper contains 11 sections, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methods
  3. Problem Formulation
  4. System Overview
  5. Automatic Patient Registration
  6. Probe Placement Guidance Generation
  7. Runtime Optimization
  8. Experiments
  9. Results and Discussion
  10. Conclusion

Figures (4)

  • Figure 1: Teleultrasound's patient-side and remote console with bidirectional streaming and AR guidance overlays via WebRTC protocol. The remote console end could monitor the examination from the headset's point-of-view camera and ultrasound video.
  • Figure 2: Overall framework of the proposed patient registration and initial probe placement guidance (PIPG). Given a batch of $K$ images, the system leverages the patient’s spatial consistency to predict the patient's anatomy and align it with the system's coordinates for guidance generation.
  • Figure 3: (a) first-person view from the AR headset during probe tracking and guidance following. (b) Predicted pose is rendered in green, guided pose is in blue, and red indicates the ground-truth pose (upper and lower sternum).
  • Figure 4: (a) Error distribution across all views in supine position. (b) and the left lateral decubitus position. Detailed statistics are summarized in Table \ref{['tab:error_stats']}.