Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robotic CBCT Meets Robotic Ultrasound

Feng Li, Yuan Bi, Dianye Huang, Zhongliang Jiang, Nassir Navab

TL;DR

The paper tackles the challenge of safely guiding needle interventions with robust multimodal imaging by introducing a robotic CBCT-US dual-modality system with one-time calibration and dynamic co-registration, enabling registration-free fusion. It combines Doppler-assisted vasculature mapping and SAM2 segmentation to project vascular information into CBCT, and implements a two-stage needle trajectory planning framework that leverages fused CBCT-US visualization. The approach demonstrates a mapping accuracy of 1.72 ± 0.62 mm and yields substantial improvements in workflow efficiency, targeting accuracy, and success rate in a user study, while maintaining registration accuracy after repositioning. These results suggest a viable path toward more automated and reliable multi-modal guidance in interventional settings, with broad potential for adoption in liver ablation and related procedures.

Abstract

The multi-modality imaging system offers optimal fused images for safe and precise interventions in modern clinical practices, such as computed tomography - ultrasound (CT-US) guidance for needle insertion. However, the limited dexterity and mobility of current imaging devices hinder their integration into standardized workflows and the advancement toward fully autonomous intervention systems. In this paper, we present a novel clinical setup where robotic cone beam computed tomography (CBCT) and robotic US are pre-calibrated and dynamically co-registered, enabling new clinical applications. This setup allows registration-free rigid registration, facilitating multi-modal guided procedures in the absence of tissue deformation. First, a one-time pre-calibration is performed between the systems. To ensure a safe insertion path by highlighting critical vasculature on the 3D CBCT, SAM2 segments vessels from B-mode images, using the Doppler signal as an autonomously generated prompt. Based on the registration, the Doppler image or segmented vessel masks are then mapped onto the CBCT, creating an optimally fused image with comprehensive detail. To validate the system, we used a specially designed phantom, featuring lesions covered by ribs and multiple vessels with simulated moving flow. The mapping error between US and CBCT resulted in an average deviation of 1.72+-0.62 mm. A user study demonstrated the effectiveness of CBCT-US fusion for needle insertion guidance, showing significant improvements in time efficiency, accuracy, and success rate. Needle intervention performance improved by approximately 50% compared to the conventional US-guided workflow. We present the first robotic dual-modality imaging system designed to guide clinical applications. The results show significant performance improvements compared to traditional manual interventions.

Robotic CBCT Meets Robotic Ultrasound

TL;DR

The paper tackles the challenge of safely guiding needle interventions with robust multimodal imaging by introducing a robotic CBCT-US dual-modality system with one-time calibration and dynamic co-registration, enabling registration-free fusion. It combines Doppler-assisted vasculature mapping and SAM2 segmentation to project vascular information into CBCT, and implements a two-stage needle trajectory planning framework that leverages fused CBCT-US visualization. The approach demonstrates a mapping accuracy of 1.72 ± 0.62 mm and yields substantial improvements in workflow efficiency, targeting accuracy, and success rate in a user study, while maintaining registration accuracy after repositioning. These results suggest a viable path toward more automated and reliable multi-modal guidance in interventional settings, with broad potential for adoption in liver ablation and related procedures.

Abstract

The multi-modality imaging system offers optimal fused images for safe and precise interventions in modern clinical practices, such as computed tomography - ultrasound (CT-US) guidance for needle insertion. However, the limited dexterity and mobility of current imaging devices hinder their integration into standardized workflows and the advancement toward fully autonomous intervention systems. In this paper, we present a novel clinical setup where robotic cone beam computed tomography (CBCT) and robotic US are pre-calibrated and dynamically co-registered, enabling new clinical applications. This setup allows registration-free rigid registration, facilitating multi-modal guided procedures in the absence of tissue deformation. First, a one-time pre-calibration is performed between the systems. To ensure a safe insertion path by highlighting critical vasculature on the 3D CBCT, SAM2 segments vessels from B-mode images, using the Doppler signal as an autonomously generated prompt. Based on the registration, the Doppler image or segmented vessel masks are then mapped onto the CBCT, creating an optimally fused image with comprehensive detail. To validate the system, we used a specially designed phantom, featuring lesions covered by ribs and multiple vessels with simulated moving flow. The mapping error between US and CBCT resulted in an average deviation of 1.72+-0.62 mm. A user study demonstrated the effectiveness of CBCT-US fusion for needle insertion guidance, showing significant improvements in time efficiency, accuracy, and success rate. Needle intervention performance improved by approximately 50% compared to the conventional US-guided workflow. We present the first robotic dual-modality imaging system designed to guide clinical applications. The results show significant performance improvements compared to traditional manual interventions.

Paper Structure

This paper contains 11 sections, 3 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methods
  3. System Calibration
  4. Vasculature Mapping Based on Doppler Imaging
  5. Needle Trajectory Planning
  6. Results
  7. CBCT-US mapping performance
  8. User study
  9. Registration Accuracy After Repositioning
  10. Discussion
  11. Conclusion

Figures (3)

  • Figure 1: Overview of the robotic CBCT-US system and fusion results.
  • Figure 2: (a) US probe positioned perpendicular to the phantom, showing the initial needle setup. (b) The US image showing the needle inserted into the phantom with the US probe perpendicular to the phantom. (c) Probe and needle positions after the robotic arm's movement. (d) US image after movement, with the needle indicating the predicted path toward the target lesion.
  • Figure 3: (a) Mapping error of blood vessels shown in color gradient. Green means the error is low, while yellow represents the error is high. (b) Mapping errors over frames.