A Framework For Automated Dissection Along Tissue Boundary

TL;DR

This work presents an end-to-end framework for automated dissection along tissue boundaries in robotic cholecystectomy, integrating AI-based tissue segmentation and keypoint detection with stereo-based 3D scene reconstruction and PBVS control. The system leverages Detectron2 trained on a bespoke ex-vivo dataset, a fiducial-marker calibration workflow, and an Arduino-enabled energy delivery interface to execute boundary-guided dissection with the da Vinci setup. Ex-vivo experiments on chicken and pig liver demonstrate submillimeter trajectory accuracy and successful autonomous energy delivery along tissue boundaries, while limitations include segmentation under atypical tissue colors and boundary-tracking robustness at image edges. The findings indicate a promising step toward autonomous dissection, with future work focusing on dataset expansion, real-time tissue boundary tracking during deformation, dual-arm manipulation, and endoscope-centered visual servoing to approach surgeon-level performance.$

Abstract

Robotic surgery promises enhanced precision and adaptability over traditional surgical methods. It also offers the possibility of automating surgical interventions, resulting in reduced stress on the surgeon, better surgical outcomes, and lower costs. Cholecystectomy, the removal of the gallbladder, serves as an ideal model procedure for automation due to its distinct and well-contrasted anatomical features between the gallbladder and liver, along with standardized surgical maneuvers. Dissection is a frequently used subtask in cholecystectomy where the surgeon delivers the energy on the hook to detach the gallbladder from the liver. Hence, dissection along tissue boundaries is a good candidate for surgical automation. For the da Vinci surgical robot to perform the same procedure as a surgeon automatically, it needs to have the ability to (1) recognize and distinguish between the two different tissues (e.g. the liver and the gallbladder), (2) understand where the boundary between the two tissues is located in the 3D workspace, (3) locate the instrument tip relative to the boundary in the 3D space using visual feedback, and (4) move the instrument along the boundary. This paper presents a novel framework that addresses these challenges through AI-assisted image processing and vision-based robot control. We also present the ex-vivo evaluation of the automated procedure on chicken and pork liver specimens that demonstrates the effectiveness of the proposed framework.

Figures (7)

• Figure 1: The overall system architecture: (a) Hardware setup. (b) Results from Detectron2, both segmentation and key point detection. (c) The extracted goal points and tooltip pose in the 3D space along with the inputs used by the control system.
• Figure 2: The setup showing how our custom-calibrated kinematics work. The transformations are shown based on the direction of the arrows and eventually, they are used to find the transformation between the ECM tip and PSM tip.
• Figure 3: Position predicted by the dVRK forward kinematics compared to the ArUco marker, and the calibrated kinematics when tested during a random motion.
• Figure 4: (a) Samples of the manually annotated segmentation dataset with SAM. (b) Samples of segmentation predictions with the trained Detectron2 model.
• Figure 5: (a) Example of the manually annotated keypoint dataset. (b) Example of keypoint predictions by the trained Detectron2 model.