Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Integrating 3D Slicer with a Dynamic Simulator for Situational Aware Robotic Interventions

Manish Sahu, Hisashi Ishida, Laura Connolly, Hongyi Fan, Anton Deguet, Peter Kazanzides, Francis X. Creighton, Russell H. Taylor, Adnan Munawar

TL;DR

This work tackles the lack of native physics simulation within 3D Slicer for image-guided robotics by integrating it with AMBF through ROS, enabling real-time, physics-based digital twins of robotic interventions. The authors implement AMBF-ROS and Slicer-ROS modules to achieve bidirectional, low-latency synchronization between AMBF's dynamic simulation and 3D Slicer's visualization/registration framework, demonstrated in a REMS ENT microsurgery use case. Key results show robust real-time performance with mean round-trip delays around 20 ms, indicating suitable responsiveness for surgical contexts. The integrated platform advances open-source tools for situational awareness in robotic surgery and lays groundwork for future assistive planning and feedback mechanisms in 3D Slicer.

Abstract

Image-guided robotic interventions represent a transformative frontier in surgery, blending advanced imaging and robotics for improved precision and outcomes. This paper addresses the critical need for integrating open-source platforms to enhance situational awareness in image-guided robotic research. We present an open-source toolset that seamlessly combines a physics-based constraint formulation framework, AMBF, with a state-of-the-art imaging platform application, 3D Slicer. Our toolset facilitates the creation of highly customizable interactive digital twins, that incorporates processing and visualization of medical imaging, robot kinematics, and scene dynamics for real-time robot control. Through a feasibility study, we showcase real-time synchronization of a physical robotic interventional environment in both 3D Slicer and AMBF, highlighting low-latency updates and improved visualization.

Integrating 3D Slicer with a Dynamic Simulator for Situational Aware Robotic Interventions

TL;DR

This work tackles the lack of native physics simulation within 3D Slicer for image-guided robotics by integrating it with AMBF through ROS, enabling real-time, physics-based digital twins of robotic interventions. The authors implement AMBF-ROS and Slicer-ROS modules to achieve bidirectional, low-latency synchronization between AMBF's dynamic simulation and 3D Slicer's visualization/registration framework, demonstrated in a REMS ENT microsurgery use case. Key results show robust real-time performance with mean round-trip delays around 20 ms, indicating suitable responsiveness for surgical contexts. The integrated platform advances open-source tools for situational awareness in robotic surgery and lays groundwork for future assistive planning and feedback mechanisms in 3D Slicer.

Abstract

Image-guided robotic interventions represent a transformative frontier in surgery, blending advanced imaging and robotics for improved precision and outcomes. This paper addresses the critical need for integrating open-source platforms to enhance situational awareness in image-guided robotic research. We present an open-source toolset that seamlessly combines a physics-based constraint formulation framework, AMBF, with a state-of-the-art imaging platform application, 3D Slicer. Our toolset facilitates the creation of highly customizable interactive digital twins, that incorporates processing and visualization of medical imaging, robot kinematics, and scene dynamics for real-time robot control. Through a feasibility study, we showcase real-time synchronization of a physical robotic interventional environment in both 3D Slicer and AMBF, highlighting low-latency updates and improved visualization.
Paper Structure (12 sections, 5 figures)

This paper contains 12 sections, 5 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methods
  4. System Design
  5. Scene Representation and Updates
  6. AMBF
  7. 3D Slicer
  8. System dependencies and compatibility
  9. Experiments
  10. Surgical robotics use case
  11. Performance Evaluation
  12. Discussion & Conclusion

Figures (5)

  • Figure 1: Overview of integration of 3D Slicer and AMBF. Different features for each software tool are highlighted around their respective logos.
  • Figure 2: Simulated robotic sinus endoscopy in AMBF (left), and real-time synchronized scene in 3D Slicer (right). The physics interaction between the robot and the anatomy in AMBF will be reflected in 3D Slicer.
  • Figure 3: Higher level structure of Slicer-ROS Module connected with AMBF Simulator. AMBF simulator and 3D slicer communicate Scene data and kinematics using ROS as an intermediary middleware.
  • Figure 4: Overview of the experimental setup for the simulated robotic sinus procedure. Galen robotics system holds an endoscope aimed at the sinus phantom (Screen A). AMBF simulator acquires motion data from the Galen robot, achieving real-time synchronization with the 3D Slicer application through the integration of a novel pipeline(Screen B).
  • Figure 5: Latency analysis: Round Time Delay(RTD) over 1000 frames