TumorMap: A Laser-based Surgical Platform for 3D Tumor Mapping and Fully-Automated Tumor Resection

TL;DR

TumorMap presents a laser-based robotic platform that integrates optical coherence tomography, laser-induced endogenous fluorescence (TumorID) for tissue diagnosis, and a noncontact fiber laser for autonomous tumor resection. The system fuses multimodal perception with deep learning-driven boundary estimation (via convex hull) and optimization-based inverse kinematics to generate precise, submillimeter laser trajectories for tumor removal. Validations span phantom phantoms, ex vivo porcine/chicken tissues, and murine models of soft tissue sarcoma and osteosarcoma, demonstrating robust 3D tumor mapping, boundary delineation, and Automated resection with minimal human intervention. A novel histopathological workflow links fluorescence signals to gold-standard pathology, establishing TumorID as a noncontact intraoperative diagnostic proxy and highlighting potential for clinical translation in superficial tumor surgery.

Abstract

Surgical resection of malignant solid tumors is critically dependent on the surgeon's ability to accurately identify pathological tissue and remove the tumor while preserving surrounding healthy structures. However, building an intraoperative 3D tumor model for subsequent removal faces major challenges due to the lack of high-fidelity tumor reconstruction, difficulties in developing generalized tissue models to handle the inherent complexities of tumor diagnosis, and the natural physical limitations of bimanual operation, physiologic tremor, and fatigue creep during surgery. To overcome these challenges, we introduce "TumorMap", a surgical robotic platform to formulate intraoperative 3D tumor boundaries and achieve autonomous tissue resection using a set of multifunctional lasers. TumorMap integrates a three-laser mechanism (optical coherence tomography, laser-induced endogenous fluorescence, and cutting laser scalpel) combined with deep learning models to achieve fully-automated and noncontact tumor resection. We validated TumorMap in murine osteoscarcoma and soft-tissue sarcoma tumor models, and established a novel histopathological workflow to estimate sensor performance. With submillimeter laser resection accuracy, we demonstrated multimodal sensor-guided autonomous tumor surgery without any human intervention.

Figures (25)

• Figure 1: Overview of the TumorMap system and the fully-automated murine tumor resection workflow. This system incorporates two sub-modules for tissue diagnosis and resection (in b) and tissue perception (in c). a: The workflow of 3D tumor map formulation. First, tumors are induced in the legs of genetically engineered mice. Once the tumor grows to the expected palpable size, the tissue of interest is dissected to extract the superficial tumor targets from the right leg while keeping the left leg as a healthy (control) sample. Based on the tumor and healthy datasets (Method. \ref{['murine_tumor_classification']}), a tumor classifier is trained to later allow tumor boundary prediction during the realistic experiment. For the offline experiment, the tumor is scanned by the OCT sensor for surface reconstruction. The laser-induced endogenous fluorescence sensor (also referred to as "TumorID", i.e., tumor identification sensor) is controlled by the robot to achieve tumor searching based on a raster scanning pattern. Each collected spectrum data is passed through the tumor classifier to generate a single-spot tumor label. This formulates a sequence of 3D tumor tags. A 3D tumor boundary is created from the tumor tags, which incorporates colorized, pathological, and geometric information, leading to the formulation of a tumor boundary based on the convex hull algorithm yap2013quantitative (Method \ref{['method_model_tumor_roi_geometry']}). Finally, the tumor boundary is post-processed to formulate a region for automated tumor removal using a fiber-coupled laser scalpel. b: The hardware setup for the tissue diagnosis and resection system. This system incorporates a 6-DOF UR5e robot arm, a laser-induced endogenous fluorescence sensor (TumorID), and a fiber-coupled cutting laser scalpel with a side-by-side configuration. The working distance between the TumorID laser to the tissue is approximately 56.3 mm. This allows for efficient implementations of robot kinematics and trajectory modeling with a single robot arm, c: The tissue perception system of a $1310~nm$ tabletop OCT system with a fixed dual-camera system relative to the surgical scene. This system is used to capture the geometric and colorized information of the murine tumor BioRender.
• Figure 2: Automated tumor resection workflow: from tumor boundary estimation to resection. a: The offline workflow of building the tumor classifiers by using the TumorID sensor. Before the online tumor resection, the tumor classifiers are trained with the TumorID datasets of the soft tissue sarcoma (STS) and osteosarcoma sarcoma (OS) murine models (Method. \ref{['method_doe_murine_robot_data_collection']}). Specifically, the dissected tumors and healthy samples are first placed at the surgical site. The TumorID is controlled by the robot to collect spectrum data following a raster scanning. The tumor classifier is represented by a multilayer perceptron model for binary classifications and used for online tumor resection (in step c). b: OCT-guided tumor reconstruction. The tumor is scanned with a tabletop OCT and a 3D surface is reconstructed from sequential B-scan images. c: Sensor information alignment from the OCT sensor and the color image through the camera-to-oct calibration in Method \ref{['method_calibration_oct_to_camera']} (red dots: points in the tumor region; yellow line: tumor boundary). The tumor boundary is overlaid with the colorized OCT map for intraoperative planning and visualization. d: 3D tumor boundary and cutting region. A sparse tumor map is converted to a 3D tumor boundary following the Method. \ref{['method_model_tumor_roi_geometry']}. A 3D laser trajectory is generated through the proposed robot kinematics solver (Method. \ref{['method_system_model_ik_and_traj_plan']}) to visit target points within the predicted tumor boundary (purple: predicted region; blue: realistic cutting region). e: The post-resection OCT surface shows unique pixel features of the cutting region after laser ablation. The distinct pixel intensities (lighter gray color) are generated due to tissue coagulation.
• Figure 3: Overview of the system accuracy quantification. These phantom experiments aim for tracking fiducial marker, 3D trajectory, and the spherical region (ROI) by using three lasers for generalization and reliability testing. Three laser modules were applied (a fiber-coupled laser with a wavelength of 1940 nm, a green laser diode with a wavelength of 650 nm, and a TumorID laser diode with a wavelength of 405 nm). a: An example of fiducial marker using the laser diode (visible green dot). The global and local views of the images show the error distribution of TumorMap to achieve end-to-end target tracking. b: An example of trajectory tracing using the green laser diode. The point-to-target error is calculated by using the correspondence index of nearest neighbors. c: The geometric relationship among the true-ROI, prediction-ROI, and actual-ROI, and the definitions of algorithm, system and calibration errors. d: The geometric relationship among the overlapping, undercutting and overcutting regions, as well as the point-to-point edge error based on the correspondence of the nearest points. e: The experimental workflow of using TumorID to determine "artificial tumor regions" and laser scalpels (laser diode and fiber-coupled laser) to trace targets (laser tools highlighted in figures). The laser diode generates a visible laser spot on the surface. This can be detected by the camera and thus its center can be accurately estimated for error calculation. The fiber-coupled laser generates a resected pattern on the surface and creates a unique feature to differentiate it from the pre-ablated surface. f: The "Marker error" graph shows the error barchart between the real-cutting and the true centers. g: The "Trajectory edge error" graph depicts the error of the point-based trajectory. h: For the subcategories, the barchart "ROI edge error" shows the point-based boundary error (step size = 1.44 mm highlighted) between the ablated region and the true one (system error: actual-to-true offset, algorithm error: predicted-to-true offset, calibration error: actual-to-predicted offset). i: The "ROI IoU" describes the ratio of the overlapping region versus the true one. j: The "ROI Undercut" shows the ratio of the region offset between the predicted region and the true region that should be cut (but has not been cut). k: The "ROI Overcut" denotes the ratio of the region offset between the predicted region and the true region that should not be cut (but has been cut). Statistical index: $*: p < 0.05$, $**: p < 0.01$, $***: p < 0.001$, $ns$: not significant).
• Figure 4: Ex vivo experiments with the laser diode and the fiber-coupled laser (unit: mm).a: The segmented reference images of the ex vivo tissue samples. b: The tumor tags with classification labels overlapped with the OCT surface reconstruction. c: The actual laser resection signatures overlapped with the post-resection image with the predicted tumor boundary. Since the laser diode does not generate cutting patterns, the laser spot is segmented in the image, transformed to the 3D OCT frame, and overlaid on the surface map. d: The algorithm evaluation for the predicted tumorous and resection regions (red: true region labeled by operators; green: predicted region from the algorithm framework; blue: realistic resection region). e: Spectrum distribution of the TumorID data with various ex vivo tissue models. The range of wavelengths is selected to show the major difference of the colorized dye to mimic the "artificial tumors" (495 nm to 570 nm for green color and 620 nm to 750 nm for red color). The intensity distribution is shown in Appendix. \ref{['appendix_exvivo_classification_graph']}. We first calculate the thresholds as the average of the mean tumor and healthy intensities within the wavelength ranges. The confidence range of the intensity is defined as the upper and lower bounds of 10% of the threshold. The average intensity higher than or lower than the confidence bounds is assigned to different classification labels. f: The distribution of the edge error for boundary tracking (system error: actual-to-true offset, algorithm error: predicted-to-true offset, calibration error: actual-to-predicted offset). g: The distribution of the intersection-over-union (IoU) for the ROI tracking task. h: The distribution of undercutting ratios for ROI tracking. i: The distribution of overcutting ratios for ROI tracking. j: Volumetric mapping of an example for visualization (chicken-red with the diode virtual resection). k: Volumetric mapping of an example for visualization (porcine-green with the fiber-coupled laser resection). Statistical index: $*: p < 0.05$, $**: p < 0.01$, $***: p < 0.001$, $ns$: not significant.
• Figure 5: Summary of murine tumor resection experiments (unit: mm).a: The segmented reference images for murine tumor models. b: Sparse tumor tags with classifications of the scanned points (tumor versus healthy). This map combines sensor information from the OCT (3D surface reconstruction), the color camera (color information), and the TumorID sensor (tumor classification). The red dot indicates the tumor tags from predictions, and the blue dot indicates the healthy one. c: 3D tumor map with highlighted boundary. The tumor boundary was formulated with the tumor tags ( from (b) ) using the convex hull algorithm to connect the edge points (Method \ref{['method_model_tumor_roi_geometry']}). d: ROI evaluations of the predicted and resection regions. The green color represents the prediction, and the blue one shows the realistic resection region. e: Histopathological analysis of resected tumor regions. As it is difficult to provide an absolute ground truth for the resection region, where the entire surface geometry changes after the histopathological processing procedure, a reference post-resection H&E slicing image is provided. This image shows that most of the cutting region covers the tumorous regions visible as discoloration of purple color (tumor). f: Volumetric rendering for the regions of prediction (green) and actual cutting (blue). g: Reference images of the post-resection mice tumors. h: ROI edge error distributions (highlighted step size = 1.86 mm). i: Ratio distributions of the Intersection-over-union, undercutting, and overcutting ratios for the laser resection tasks.