Towards Fluorescence-Guided Autonomous Robotic Partial Nephrectomy on Novel Tissue-Mimicking Hydrogel Phantoms

TL;DR

This work tackles autonomous robotic partial nephrectomy by fusing fluorescence-guided tumor segmentation with margin-aware incision planning, validated on novel tissue-mimicking hydrogel kidney phantoms. It introduces fluorescence-enabled phantoms derived from patient data, enabling realistic visualization and electrosurgical interaction, and pairs them with a perception pipeline that labels tissue via point-cloud data using SAM2 segmentation. The system demonstrates autonomous planning and execution of incisions around irregular tumor geometries with an average margin accuracy of $1.44$ mm and an average completion time of $69$ s across four phantoms. While promising, the study discusses translational limitations and outlines future work toward MIS integration and multi-view scene reconstruction to better handle real-world anatomy and occlusions.

Abstract

Autonomous robotic systems hold potential for improving renal tumor resection accuracy and patient outcomes. We present a fluorescence-guided robotic system capable of planning and executing incision paths around exophytic renal tumors with a clinically relevant resection margin. Leveraging point cloud observations, the system handles irregular tumor shapes and distinguishes healthy from tumorous tissue based on near-infrared imaging, akin to indocyanine green staining in partial nephrectomy. Tissue-mimicking phantoms are crucial for the development of autonomous robotic surgical systems for interventions where acquiring ex-vivo animal tissue is infeasible, such as cancer of the kidney and renal pelvis. To this end, we propose novel hydrogel-based kidney phantoms with exophytic tumors that mimic the physical and visual behavior of tissue, and are compatible with electrosurgical instruments, a common limitation of silicone-based phantoms. In contrast to previous hydrogel phantoms, we mix the material with near-infrared dye to enable fluorescence-guided tumor segmentation. Autonomous real-world robotic experiments validate our system and phantoms, achieving an average margin accuracy of 1.44 mm in a completion time of 69 sec.

Figures (7)

• Figure 1: Patient CT data is segmented to create volumetric meshes of the kidney, embedded tumor, and the renal hilum. After smoothing the meshes, the negatives of the volumetric meshes are 3D printed to serve as molds for our hydrogel phantoms.
• Figure 2: Hydrogel phantoms are compatible with electrosurgical instruments, such as cauterization electrodes, providing more realistic simulation capabilities compared to silicone phantoms. This enables autonomous robotic electrosurgical incisions.
• Figure 3: We create four hydrogel phantoms that cover different tumor sizes and anatomical sites. The healthy tissue is mixed with a fluorescent dye to differentiate between healthy and tumorous tissue replicating negative staining . The image is segmented into background, healthy tissue, and tumor. The labels are subsequently used to segment the point cloud observation. Based on the segmented point cloud, we plan a robotic incision path that respects a surgical margin around the tumor. After incision, we capture a CT scan of the phantom to evaluate incision accuracy.
• Figure 4: Experimental results for anatomical accuracy (column 1) and image segmentation (columns 2 and 3).
• Figure 5: We create samples with different dye concentrations to evaluate the quality of the NIR signal. The correlation between dye concentration and sbr is approximately linear. In cool and shielded storage conditions, the visibility of the dye degrades on average 29.70% over seven days.