Tracking Tumors under Deformation from Partial Point Clouds using Occupancy Networks

TL;DR

The findings indicate that the proposed method can localize tumors in moderately deforming kidneys with a margin of 6mm to 10mm, while providing essential volumetric 3D information at over 60Hz, which directly enables downstream tasks such as robotic resection.

Abstract

To track tumors during surgery, information from preoperative CT scans is used to determine their position. However, as the surgeon operates, the tumor may be deformed which presents a major hurdle for accurately resecting the tumor, and can lead to surgical inaccuracy, increased operation time, and excessive margins. This issue is particularly pronounced in robot-assisted partial nephrectomy (RAPN), where the kidney undergoes significant deformations during operation. Toward addressing this, we introduce a occupancy network-based method for the localization of tumors within kidney phantoms undergoing deformations at interactive speeds. We validate our method by introducing a 3D hydrogel kidney phantom embedded with exophytic and endophytic renal tumors. It closely mimics real tissue mechanics to simulate kidney deformation during in vivo surgery, providing excellent contrast and clear delineation of tumor margins to enable automatic threshold-based segmentation. Our findings indicate that the proposed method can localize tumors in moderately deforming kidneys with a margin of 6mm to 10mm, while providing essential volumetric 3D information at over 60Hz. This capability directly enables downstream tasks such as robotic resection.

Figures (6)

• Figure 1: Given a known kidney phantom with two tumors and a pre-operative CT scan, we estimate, from a single sensor point cloud derived from a depth image, a dense occupancy point cloud that encodes the locations of each tumor.
• Figure 2: Our processing pipeline used to perform an interactive tumor localization and robotic resection in simulation. There are two processing blocks. The Pre-Operative part uses a model obtained from an imaging modality, such as a CT image, to produce training data for our occupancy network. During training, the occupancy network learns to estimate the locations and shapes of the tumors under deformations. The Intra-Operative part uses a sensor point cloud obtained from an RGBD camera to output an occupancy point cloud that is innately compatible with our surgical resection method. The visualizations for the occupancy networks are 2D simplifications.
• Figure 3: The kidney phantom is placed in the supine position onto the CT machine gantry. The RGBD camera is set up $15\text{cm}$ above the phantom.
• Figure 4: From real patient CT cases, we segmented the patient's kidney and tumors, generated a CAD model, and 3D printed the phantom mold. By injecting contrast-enhanced PVA liquid, we created a hydrogel phantom that exhibits clear tumor and parenchyma visualization under CT imaging.
• Figure 5: Figure (a),(b) and (c) depict the kidney phantom in a normal supine configuration with a midsection height of approximately $57.87\text{mm}$. Figures (d), (e) and (f) present the measurements of the midsection under compression, which are $51.21\text{mm}$, $55.00\text{mm}$, $59.06\text{mm}$