Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Realistic Surgical Simulator for Non-Rigid and Contact-Rich Manipulation in Surgeries with the da Vinci Research Kit

Yafei Ou, Sadra Zargarzadeh, Paniz Sedighi, Mahdi Tavakoli

TL;DR

The paper addresses the need for realistic, contact-rich surgical simulators capable of handling rigid instruments, soft tissue, fluids, and cloth in a unified environment for robotics research. It introduces CRESSim, a Unity and Nvidia PhysX 5-based platform that integrates the real dVRK via ROS for VR teleoperation and supports new instruments and tasks. Key contributions include a native PhysX 5 plugin for Unity, a GUI-based scene editor, two new instruments, and three demonstration tasks (tissue grasping, blood suction, tissue cutting) with real-time performance. The platform has potential to advance research in surgical automation and learning, and the authors plan to release the source code to the community for broader use and development.

Abstract

Realistic real-time surgical simulators play an increasingly important role in surgical robotics research, such as surgical robot learning and automation, and surgical skills assessment. Although there are a number of existing surgical simulators for research, they generally lack the ability to simulate the diverse types of objects and contact-rich manipulation tasks typically present in surgeries, such as tissue cutting and blood suction. In this work, we introduce CRESSim, a realistic surgical simulator based on PhysX 5 for the da Vinci Research Kit (dVRK) that enables simulating various contact-rich surgical tasks involving different surgical instruments, soft tissue, and body fluids. The real-world dVRK console and the master tool manipulator (MTM) robots are incorporated into the system to allow for teleoperation through virtual reality (VR). To showcase the advantages and potentials of the simulator, we present three examples of surgical tasks, including tissue grasping and deformation, blood suction, and tissue cutting. These tasks are performed using the simulated surgical instruments, including the large needle driver, suction irrigator, and curved scissor, through VR-based teleoperation.

A Realistic Surgical Simulator for Non-Rigid and Contact-Rich Manipulation in Surgeries with the da Vinci Research Kit

TL;DR

The paper addresses the need for realistic, contact-rich surgical simulators capable of handling rigid instruments, soft tissue, fluids, and cloth in a unified environment for robotics research. It introduces CRESSim, a Unity and Nvidia PhysX 5-based platform that integrates the real dVRK via ROS for VR teleoperation and supports new instruments and tasks. Key contributions include a native PhysX 5 plugin for Unity, a GUI-based scene editor, two new instruments, and three demonstration tasks (tissue grasping, blood suction, tissue cutting) with real-time performance. The platform has potential to advance research in surgical automation and learning, and the authors plan to release the source code to the community for broader use and development.

Abstract

Realistic real-time surgical simulators play an increasingly important role in surgical robotics research, such as surgical robot learning and automation, and surgical skills assessment. Although there are a number of existing surgical simulators for research, they generally lack the ability to simulate the diverse types of objects and contact-rich manipulation tasks typically present in surgeries, such as tissue cutting and blood suction. In this work, we introduce CRESSim, a realistic surgical simulator based on PhysX 5 for the da Vinci Research Kit (dVRK) that enables simulating various contact-rich surgical tasks involving different surgical instruments, soft tissue, and body fluids. The real-world dVRK console and the master tool manipulator (MTM) robots are incorporated into the system to allow for teleoperation through virtual reality (VR). To showcase the advantages and potentials of the simulator, we present three examples of surgical tasks, including tissue grasping and deformation, blood suction, and tissue cutting. These tasks are performed using the simulated surgical instruments, including the large needle driver, suction irrigator, and curved scissor, through VR-based teleoperation.
Paper Structure (20 sections, 8 figures, 2 tables, 2 algorithms)

This paper contains 20 sections, 8 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Surgery and Surgical Robot Simulation
  4. Simulation for Non-rigid and Contact-Rich Manipulation
  5. System Architecture
  6. Overview
  7. Physics Functionalities
  8. Rigid body and robot
  9. FEM Soft body
  10. PBD Cloth and Fluid
  11. Manipulation and Contact Features
  12. Teleoperation in Virtual Reality
  13. Example Surgical Scenes
  14. Simulated PSM with Various Surgical Instruments
  15. Simulated Contact-Rich Surgical Tasks
  16. ...and 5 more sections

Figures (8)

  • Figure 1: Teleoperating the CRESSim environment using the dVRK console.
  • Figure 2: System architecture.
  • Figure 3: Inspector window for defining an FEM soft body.
  • Figure 4: Examples of simulated objects. (a) Rigid and soft bodies; (b) Cloth with fixed vertices; (c) Fluid.
  • Figure 5: Simulated PSM. (a) PSM robot; (b) Large needle driver; (c) Suction irrigator; (d) Curved scissor.
  • ...and 3 more figures