Give me scissors: Collision-Free Dual-Arm Surgical Assistive Robot for Instrument Delivery

TL;DR

A vision-language model is utilized to automatically generate the robot's grasping and delivery trajectories in a zero-shot manner based on surgeons' instructions, and a real-time obstacle minimum distance perception method is proposed and integrated into a unified quadratic programming framework.

Abstract

During surgery, scrub nurses are required to frequently deliver surgical instruments to surgeons, which can lead to physical fatigue and decreased focus. Robotic scrub nurses provide a promising solution that can replace repetitive tasks and enhance efficiency. Existing research on robotic scrub nurses relies on predefined paths for instrument delivery, which limits their generalizability and poses safety risks in dynamic environments. To address these challenges, we present a collision-free dual-arm surgical assistive robot capable of performing instrument delivery. A vision-language model is utilized to automatically generate the robot's grasping and delivery trajectories in a zero-shot manner based on surgeons' instructions. A real-time obstacle minimum distance perception method is proposed and integrated into a unified quadratic programming framework. This framework ensures reactive obstacle avoidance and self-collision prevention during the dual-arm robot's autonomous movement in dynamic environments. Extensive experimental validations demonstrate that the proposed robotic system achieves an 83.33% success rate in surgical instrument delivery while maintaining smooth, collision-free movement throughout all trials. The project page and source code are available at https://give-me-scissors.github.io/.

Figures (10)

• Figure 1: Surgical instruments transfer process. (a) Scrub nurse instrument delivery. (b) Robot-assisted instrument delivery.
• Figure 2: The pipeline of the collision-free dual-arm surgical assistive robot for instrument delivery. The robot system receives multi-modal inputs from the physical world (i.e., surgeon instruction, color image, depth data). The real-time obstacle perception module computes the minimum distance between robot and environmental obstacles. The QP framework is built upon the minimum distance, ensuring the dual-arm robot's collision-free operation. The high-level task planning utilizes VLM to generate the desired motion objectives for the QP framework.
• Figure 3: Real-time obstacle perception process. The robot joint configuration and point cloud of obstacles are taken as input. Output is the minimum distance between robot and obstacle.
• Figure 4: Obstacle and self-collision avoidance process of the dual-arm robot in simulation. The red arrows indicate the avoidance directions for dual-arm robot. (a) Self-collision avoidance. (b) Obstacle avoidance.
• Figure 5: The trajectories of dual-arm robot's end-effector and obstacles during simulation. The avoidance motion is achieved by our method.