Confidence-based Intent Prediction for Teleoperation in Bimanual Robotic Suturing

TL;DR

This work tackles the challenges of traditional teleoperation and full autonomy in robotic suturing by introducing a unified framework that combines real-time Surgeme prediction with confidence-guided low-level control. It integrates a Transformer-based gesture classifier to infer high-level surgeon intent and a Bayesian, confidence-aware IAC to blend human and robot targets, implemented on the da Vinci Research Kit with CHENA for improved needle handling. Validation includes the STITCHES dataset and two user studies showing faster task completion, reduced perpendicularity errors, and lower perceived workload compared with traditional teleoperation. The results demonstrate the feasibility and practical value of real-time intent-aware shared control for bimanual robotic suturing, with potential applicability to broader teleoperation domains.

Abstract

Robotic-assisted procedures offer enhanced precision, but while fully autonomous systems are limited in task knowledge, difficulties in modeling unstructured environments, and generalisation abilities, fully manual teleoperated systems also face challenges such as delay, stability, and reduced sensory information. To address these, we developed an interactive control strategy that assists the human operator by predicting their motion plan at both high and low levels. At the high level, a surgeme recognition system is employed through a Transformer-based real-time gesture classification model to dynamically adapt to the operator's actions, while at the low level, a Confidence-based Intention Assimilation Controller adjusts robot actions based on user intent and shared control paradigms. The system is built around a robotic suturing task, supported by sensors that capture the kinematics of the robot and task dynamics. Experiments across users with varying skill levels demonstrated the effectiveness of the proposed approach, showing statistically significant improvements in task completion time and user satisfaction compared to traditional teleoperation.

Figures (7)

• Figure 1: Bimanual robotic suturing with four key surgemes: (1) Needle tip positioned at the entry point; (2) Needle pushed/inserted reaching the exit point; (3) Needle pulled out; (4) Needle passed to the starting manipulator.
• Figure 2: Design of the Compact Holder for Enhanced Needle Alignment (CHENA). (a-b) The structure comprises of walls that create a concavity where the needle is attracted and maintained fixed by a magnet. (c-d) The needle self-aligns using to two independent acting forces: the jaw pushing and the magnet pulling the needle towards the cavity.
• Figure 3: Comparison between a) traditional and b) C-IAC teleoperation. While traditional teleoperation has inherent delay to reach the current position of the user's hand, C-IAC directly moves towards the predicted position. $\lambda$ is used in the shared control scheme to adjust the assistance. With high confidence ($\lambda\approx1$), delay is minimized as the robot to follow its own target.
• Figure 4: Task frame position on the tissue surface using a ChArUco with x-axis parallel to the wound, y-axis perpendicular to it, and z-axis perpendicular to the tissue surface. During suturing, the robot refers to the task frame Cartesian coordinates to determine wound orientation and insertion point.
• Figure 5: Bimanual suturing control scheme using C-IAC and Transformer for gesture classification. Sensory information from markers and robot kinematics undergoes Bayesian inference to compute the confidence parameter $\lambda$ for the IAC. The Transformer utilizes the kinematics to classify the user’s gesture and define the target $\tau_r$. In IAC, the user input $u_h$ is used to predict the human's target $\tau_h$, which, together with $\tau_r$, determines the robot manipulator's target.