Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cardiac Copilot: Automatic Probe Guidance for Echocardiography with World Model

Haojun Jiang, Zhenguo Sun, Ning Jia, Meng Li, Yu Sun, Shaqi Luo, Shiji Song, Gao Huang

TL;DR

The core innovation lies in proposing a data-driven world model, named Cardiac Dreamer, for representing cardiac spatial structures, which can provide structure features of any cardiac planes around the current probe position in the latent space, serving as an precise navigation map for autonomous plane localization.

Abstract

Echocardiography is the only technique capable of real-time imaging of the heart and is vital for diagnosing the majority of cardiac diseases. However, there is a severe shortage of experienced cardiac sonographers, due to the heart's complex structure and significant operational challenges. To mitigate this situation, we present a Cardiac Copilot system capable of providing real-time probe movement guidance to assist less experienced sonographers in conducting freehand echocardiography. This system can enable non-experts, especially in primary departments and medically underserved areas, to perform cardiac ultrasound examinations, potentially improving global healthcare delivery. The core innovation lies in proposing a data-driven world model, named Cardiac Dreamer, for representing cardiac spatial structures. This world model can provide structure features of any cardiac planes around the current probe position in the latent space, serving as an precise navigation map for autonomous plane localization. We train our model with real-world ultrasound data and corresponding probe motion from 110 routine clinical scans with 151K sample pairs by three certified sonographers. Evaluations on three standard planes with 37K sample pairs demonstrate that the world model can reduce navigation errors by up to 33\% and exhibit more stable performance.

Cardiac Copilot: Automatic Probe Guidance for Echocardiography with World Model

TL;DR

The core innovation lies in proposing a data-driven world model, named Cardiac Dreamer, for representing cardiac spatial structures, which can provide structure features of any cardiac planes around the current probe position in the latent space, serving as an precise navigation map for autonomous plane localization.

Abstract

Echocardiography is the only technique capable of real-time imaging of the heart and is vital for diagnosing the majority of cardiac diseases. However, there is a severe shortage of experienced cardiac sonographers, due to the heart's complex structure and significant operational challenges. To mitigate this situation, we present a Cardiac Copilot system capable of providing real-time probe movement guidance to assist less experienced sonographers in conducting freehand echocardiography. This system can enable non-experts, especially in primary departments and medically underserved areas, to perform cardiac ultrasound examinations, potentially improving global healthcare delivery. The core innovation lies in proposing a data-driven world model, named Cardiac Dreamer, for representing cardiac spatial structures. This world model can provide structure features of any cardiac planes around the current probe position in the latent space, serving as an precise navigation map for autonomous plane localization. We train our model with real-world ultrasound data and corresponding probe motion from 110 routine clinical scans with 151K sample pairs by three certified sonographers. Evaluations on three standard planes with 37K sample pairs demonstrate that the world model can reduce navigation errors by up to 33\% and exhibit more stable performance.
Paper Structure (15 sections, 7 equations, 3 figures, 1 table)

This paper contains 15 sections, 7 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Method
  3. Target-oriented Guidance Framework
  4. Foresee States by Cardiac Dreamer
  5. Experiments
  6. Setup
  7. Implementation Details
  8. Evaluation Metrics
  9. Results and Analysis
  10. Accuracy.
  11. Stability.
  12. Analysis.
  13. Conclusion
  14. Acknowledgement.
  15. Disclosure of Interests.

Figures (3)

  • Figure 1: Schematic diagram of cardiac copilot system.(a1-2) We collect a large-scale expert demonstration dataset with extensive ultrasound image and probe motion sample pairs using a robotic arm. The demonstration data contains critical medical knowledge required for performing echocardiography. (b) We encapsulate experts’ knowledge within deep neural networks to facilitate automatic probe guidance.
  • Figure 2: Diagram of target-oriented guidance framework.Left is the policy network which provides the basic guidance signal for locating the target plane. Right is the Cardiac Dreamer which foresees states reached by executing actions output by the policy network and refines the actions based on these states. The details of the action combining operator is shown in Eq. \ref{['eq:aciton_combine']}.
  • Figure 3: Detailed analysis. For optimal viewing, please consider enlarging and viewing in color. The change of predicted absolute error of different samples with respect to the mean pose difference to target plane on three standard planes. The 'mean pose difference' refers to calculating the mean absolute error between the pose of the current plane and that of the target plane. The dashed line represents the mean value, and the shading represents the standard deviation.