Autonomous Image-to-Grasp Robotic Suturing Using Reliability-Driven Suture Thread Reconstruction

TL;DR

The paper addresses autonomous suturing in RAMIS by tackling the twin challenges of noisy endoscopic suture observations and reliable thread grasping. It introduces a reliability-aware image-to-grasp pipeline that reconstructs a smooth thread spline via an iterative quadratic-programming approach, enforcing bounds derived from observation noise, and couples it with a capture-slide-grasp strategy that maximizes the probability of successful grasping. The reconstruction leverages a minimum-variation criterion on a cubic B-spline under a constant-speed parameterization, reducing the objective to a quadratic form and solving a sequence of linearly constrained QPs to converge within a few iterations. Experimental validation on the dVRK system across multiple thread configurations and backgrounds demonstrates state-of-the-art reconstruction accuracy and a robust grasping policy, achieving up to 97.0% success with reliability-driven planning. This work advances autonomous suturing by enabling entanglement-averse grasping trajectories and lays groundwork for integration with knot-tying and other autonomous suture manipulation tasks.

Abstract

Automating suturing during robotically-assisted surgery reduces the burden on the operating surgeon, enabling them to focus on making higher-level decisions rather than fatiguing themselves in the numerous intricacies of a surgical procedure. Accurate suture thread reconstruction and grasping are vital prerequisites for suturing, particularly for avoiding entanglement with surgical tools and performing complex thread manipulation. However, such methods must be robust to heavy perceptual degradation resulting from heavy noise and thread feature sparsity from endoscopic images. We develop a reconstruction algorithm that utilizes quadratic programming optimization to fit smooth splines to thread observations, satisfying reliability bounds estimated from measured observation noise. Additionally, we craft a grasping policy that generates gripper trajectories that maximize the probability of a successful grasp. Our full image-to-grasp pipeline is rigorously evaluated with over 400 grasping trials, exhibiting state-of-the-art accuracy. We show that this strategy can be applied to the various techniques in autonomous suture needle manipulation to achieve autonomous surgery in a generalizable way.

Figures (6)

• Figure 1: Suture thread reconstruction and grasping are key prerequisites for autonomous suturing, but are difficult to perform due to heavy noise and thread feature sparsity in surgical endoscopic images. We develop a reliability-driven suture thread image-to-grasp pipeline that generates spline reconstructions with attached confidence (where low-to-high confidence is highlighted by the red-to-green spectrum displayed) and executes gripper trajectories that maximize the probability of a successful grasp.
• Figure 2: Our full image-to-grasp pipeline: a) Segment the suture thread (cyan) from rectified stereo images. b) Use outlier rejection and clustering to collect thread observations, each with an associated reliability region encoding uncertainty. c) Execute a quadratic-programming-based minimum variation spline reconstruction algorithm to produce a smooth, realistic spline passing through all reliability regions. d) Utilize reliability regions to estimate reconstruction confidence and plan gripper trajectories, maximizing the chance of a successful grasp.
• Figure 3: (a) Due to observation noise, some portions of a thread reconstruction may be inaccurate, preventing direct grasping. (b) To mitigate this, we can use estimated reconstruction confidence to initially "capture" the thread at a reliable point and "slide" the gripper along our reconstruction to our previously inaccurate grasping point, lightly manipulating the thread such that it stays within the gripper jaws.
• Figure 4: In this toy two-dimensional example with 3 reliability regions, we can construct two splines, $B_1(s)$ and $B_2(s)$, that pass through all regions. Let $s_1, s_2,$ and $s_3$ be a fixed set of parameters for which the reliability-region constraints are met for $B_1(s)$. If we evaluate $B_2(s)$ at these same $s_j$ values, we find that $B(s_2)$ and $B(s_3)$ lie outside the reliability regions. Hence, $s_j$ values must change when generating different splines. We adopt an iterative spline-fitting approach that allows the $s_j$ parameters to change between each iteration.
• Figure 5: We crafted 10 evaluative scenarios, consisting of 5 thread configuration types with each repeated on 2 backgrounds. The 3rd-person and endoscopic views and reconstructions of 3 particular scenarios are displayed here: (a) the Medium configuration with chicken background which tests for moderate thread alignment with epipolar lines, (b) the Singularity configuration with surgical paper background which tests for thread alignment with the endoscope's $z$-axis, and (c) the Occlusion scenario with chicken background which tests for tool occlusion. Our image-to-grasp pipeline not only produces accurate reconstructions but also correctly estimates reconstruction confidence based on each perceptual degradation case.