Force-Aware Autonomous Robotic Surgery

TL;DR

This work tackles the challenge of force-aware autonomy for the third arm in robot-assisted surgery by incorporating tool-tissue interaction forces into imitation-learned policies. Using an adapted Action Chunking Transformer (ACT) with a CVAE-based policy, it compares force-enabled and force-free variants trained on 60 demonstrations with a dVRK setup and evaluated on seen and unseen tissue. The force-aware policy achieves substantially higher success rates (threefold on seen tissue and 3.5-fold on unseen tissue) while applying significantly gentler forces, demonstrating better generalization and adherence to tissue-handling guidelines. These results indicate that incorporating force feedback into autonomous surgical policies can enhance robustness and safety, and align with force-feedback-enabled platforms like the da Vinci 5.

Abstract

This work demonstrates the benefits of using tool-tissue interaction forces in the design of autonomous systems in robot-assisted surgery (RAS). Autonomous systems in surgery must manipulate tissues of different stiffness levels and hence should apply different levels of forces accordingly. We hypothesize that this ability is enabled by using force measurements as input to policies learned from human demonstrations. To test this hypothesis, we use Action-Chunking Transformers (ACT) to train two policies through imitation learning for automated tissue retraction with the da Vinci Research Kit (dVRK). To quantify the effects of using tool-tissue interaction force data, we trained a "no force policy" that uses the vision and robot kinematic data, and compared it to a "force policy" that uses force, vision and robot kinematic data. When tested on a previously seen tissue sample, the force policy is 3 times more successful in autonomously performing the task compared with the no force policy. In addition, the force policy is more gentle with the tissue compared with the no force policy, exerting on average 62% less force on the tissue. When tested on a previously unseen tissue sample, the force policy is 3.5 times more successful in autonomously performing the task, exerting an order of magnitude less forces on the tissue, compared with the no force policy. These results open the door to design force-aware autonomous systems that can meet the surgical guidelines for tissue handling, especially using the newly released RAS systems with force feedback capabilities such as the da Vinci 5.

Figures (5)

• Figure 1: The CVAE decoder used to generate the learned policy in the modified ACT architecture. The CVAE decoder synthesizes the stereoscopic images, concatenated robot joints and force/torque data, and latent variable $z$ with a transformer encoder. Then, it predicts the robot's action sequence with a transformer decoder.
• Figure 2: (a) The hardware setup used in this work. The tissue sample is mounted on a rigid board. The force/torque sensor is mounted under the board and is placed on a flat surface. This figure shows the tissue being lifted as the final part of the tissue retraction task. (b) One of the tissue samples used in this work. The task is to retract the triangular area to uncover the area underneath it.
• Figure 3: Data Collection Scheme: (a) Visualization of smoothed force values. The raw force/torque data was smoothed by applying an averaging window to reduce the effect of the measurement noise. (b) Visualization of data collection of multiple modalities. All modalities were recorded at the camera frequency: 30 Hz. The purple dots indicate the force/torque values (smoothed) and joint kinematics recorded at time steps N-1, N, and N+1.
• Figure 4: (a) Average total applied force and standard deviation for both policy roll outs on the seen tissue sample. (b) Normalized histogram of forces showing the total time a specific force is applied during both policy roll outs on the seen tissue sample. (c) Histogram of differences of applied force during both policy roll outs on the seen tissue sample.
• Figure 5: (a) Average total applied force and standard deviation for both policy roll outs on the unseen tissue sample. (b) Normalized histogram of forces showing the total time a specific force is applied during both policy roll outs on the unseen tissue sample. (c) Histogram of differences of applied force during both policy roll outs on the unseen tissue sample.