Surgical Gym: A high-performance GPU-based platform for reinforcement learning with surgical robots

TL;DR

The paper addresses the data-hungry nature of reinforcement learning for autonomous surgical robots by introducing Surgical Gym, a GPU-based, tensorized simulator that runs physics, observations, rewards, and policy optimization entirely on the GPU. It presents five tasks across six robots (including dVRK PSM/ECM and STAR) and demonstrates substantial speedups over CPU-based platforms, achieving up to $2.9$ s per $1$M timesteps without learning and $6.8$ s per $1$M timesteps with learning, while supporting up to 20,000 parallel environments on a single GPU. The system leverages the Isaac Gym framework, URDF/USD robot descriptions, and a GPU-optimized PPO, enabling rapid data collection and training for surgical robotics RL. The results suggest Surgical Gym can democratize access to large-scale surgical training data and pave the way for more autonomous robotic surgery, with future work targeting soft-tissue modeling and smoother sim-to-hardware transfer. Overall, this work delivers a high-performance, open-source platform that significantly lowers the barrier to developing and evaluating RL-based surgical automation.

Abstract

Recent advances in robot-assisted surgery have resulted in progressively more precise, efficient, and minimally invasive procedures, sparking a new era of robotic surgical intervention. This enables doctors, in collaborative interaction with robots, to perform traditional or minimally invasive surgeries with improved outcomes through smaller incisions. Recent efforts are working toward making robotic surgery more autonomous which has the potential to reduce variability of surgical outcomes and reduce complication rates. Deep reinforcement learning methodologies offer scalable solutions for surgical automation, but their effectiveness relies on extensive data acquisition due to the absence of prior knowledge in successfully accomplishing tasks. Due to the intensive nature of simulated data collection, previous works have focused on making existing algorithms more efficient. In this work, we focus on making the simulator more efficient, making training data much more accessible than previously possible. We introduce Surgical Gym, an open-source high performance platform for surgical robot learning where both the physics simulation and reinforcement learning occur directly on the GPU. We demonstrate between 100-5000x faster training times compared with previous surgical learning platforms. The code is available at: https://github.com/SamuelSchmidgall/SurgicalGym.

Figures (3)

• Figure 1: Demonstration of Smart Tissue Autonomous Robot robot performing path following (path in blue dashed lines) going from right to left. Suturing tool end-effector (red) precisely follows randomly generated points along a nonlinear path.
• Figure 2: (Left) Data process flow comparison. Comparison between the traditional reinforcement learning pipeline which use CPU based physics engines and Surgical Gym, which uses a GPU based physics engine. Surgical Gym avoids expensive data transfer between the CPU and GPU, with all operations tensorized on the GPU. (Right) Detailed Surgical Gym process flow. Large scale GPU accelerated physics are executed with thousands of robots in parallel. Sensor data is calculated from actor bodies and their corresponding positions, rotations and velocities which are stored directly in PyTorch tensors. Reward calculation takes relevant sensor data, calculates a per-agent reward, which is passed to the GPU-accelerated Proximal Policy Optimization implementation for gradient calculation. Finally, sensor data is forward propagated through an ANN to calculate an action, which is sent to the physics engine.
• Figure 3: Demonstration of surgical robots & attachments supported in Surgical Gym on a target reaching task.