STITCH 2.0: Extending Augmented Suturing with EKF Needle Estimation and Thread Management

TL;DR

STITCH 2.0 addresses the problem of variability in suturing quality by augmenting surgeons with a robotic pipeline that autonomously executes multiple sutures. It introduces a $6D$ EKF needle pose estimator, a $3D$ suture alignment module, flood-filled stereo processing, skeletonized U-Net masking, and an Augmented Dexterity Controller to coordinate two grippers. Across 15 trials, it achieved a wound-closure rate of 74.4 percent with an average of 4.87 sutures per trial, corresponding to 66 percent more sutures in 38 percent less time compared with STITCH 1.0; with up to two human interventions, it reached six sutures with 100 percent wound closure. These results demonstrate the viability of augmented suturing with improved perception, thread management, and precise needle handling for potential clinical utility.

Abstract

Surgical suturing is a high-precision task that impacts patient healing and scarring. Suturing skill varies widely between surgeons, highlighting the need for robot assistance. Previous robot suturing works, such as STITCH 1.0 [1], struggle to fully close wounds due to inaccurate needle tracking and poor thread management. To address these challenges, we present STITCH 2.0, an elevated augmented dexterity pipeline with seven improvements including: improved EKF needle pose estimation, new thread untangling methods, and an automated 3D suture alignment algorithm. Experimental results over 15 trials find that STITCH 2.0 on average achieves 74.4% wound closure with 4.87 sutures per trial, representing 66% more sutures in 38% less time compared to the previous baseline. When two human interventions are allowed, STITCH 2.0 averages six sutures with 100% wound closure rate. Project website: https://stitch-2.github.io/

Figures (10)

• Figure 1: Augmented Suturing: We use the da Vinci Research Kit, equipped with two grippers $g_{1}$ and $g_{2}$. STITCH 1.0 averages 2.93 sutures that were untightened and did not fully close the wound. STITCH 2.0 improved to 4.87 sutures, delivering tight, even sutures that fully close the wound.
• Figure 2: Overview of the STITCH 2.0 pipeline with extensions from STITCH 1.0 hari2024stitch in green. STITCH 2.0 integrates a 3D Suture Alignment, 6D EKF Needle Pose Estimator, and Augmented Dexterity controller. The 3D suture alignment determines optimal insertion points, the 6D EKF Pose Estimator provides geometric data for precise needle handling, and the Augmented Dexterity controller executes suturing.
• Figure 3: The 3D Suture Alignment starts with generating a scene pointcloud using RAFT-Stereo lipson2021raft, followed by segmentation of the wound center, surface, and phantom with Segment Anything (SAM) kirillov2023segment. The wound height is calculated by measuring the distance between the wound and phantom surfaces through RANSAC fitting. A 3D line representing the wound center is obtained by projecting points onto the top wound surface plane and fitting with RANSAC. Suture positions are evenly distributed along the centerline, and the wound width and height are used to determine the insertion points for needle placement.
• Figure 4: The 6D Needle Pose Estimator begins by training a U-Net for needle segmentation. A flood-fill approach is applied to reduce noise and eliminate specularities before processing through RAFT-Stereo lipson2021raft to generate a disparity image. The disparity image is converted into a depth map, and a skeletonized mask is used to improve pose estimates and accelerate circle fitting. The pose is further stabilized with an Extended Kalman Filter (EKF). Needle tip estimation is refined by adaptive thresholding and computing the intersection of the pixel ray with the EKF 3D circle estimate, enabling precise needle manipulation for insertion and suture alignment. New contributions with STITCH 2.0 from STITCH 1.0 hari2024stitch are in green.
• Figure 5: The Augmented Dexterity Controller enables suturing automation through five components: (a), (b) needle insertion, (c), (d) thread sweeping, (e), (f) needle extraction with thread cinching, (g)-(i) needle alignment and handover, and (j)-(l) pre-insertion alignment. Needle insertion involves precise suture positioning. Thread sweeping prevents tangling, while extraction involves cinching for wound closure. Needle alignment and handover correct needle orientation, and pre-insertion alignment facilitates a high-quality insertion. New STITCH 2.0 contributions from STITCH 1.0 hari2024stitch are in green.