Initial Study On Improving Segmentation By Combining Preoperative CT And Intraoperative CBCT Using Synthetic Data

TL;DR

The paper tackles segmentation under imperfect registration between intraoperative CBCT and preoperative CT, using synthetic data to simulate varying CBCT quality. It proposes a multimodal early-fusion pipeline that concatenates intraoperative CBCT with a misaligned preoperative CT as a four-channel input to a 3D U-Net, trained with binary cross-entropy and Dice loss. Evaluations on the CBCTLiTS dataset across undersampling factors $α_{np}$ and misalignment factors $α_a$ show improvements in 18 of 20 configurations, with liver Dice increasing from 0.78 to 0.88 and liver-tumor Dice from 0.03 to 0.17 at high degradation ($α_{np}=490$). The results suggest robustness of multimodal fusion to CBCT artifacts and misalignment, with potential extensions to other architectures and modalities for intraoperative imaging tasks.

Abstract

Computer-Assisted Interventions enable clinicians to perform precise, minimally invasive procedures, often relying on advanced imaging methods. Cone-beam computed tomography (CBCT) can be used to facilitate computer-assisted interventions, despite often suffering from artifacts that pose challenges for accurate interpretation. While the degraded image quality can affect image analysis, the availability of high quality, preoperative scans offers potential for improvements. Here we consider a setting where preoperative CT and intraoperative CBCT scans are available, however, the alignment (registration) between the scans is imperfect to simulate a real world scenario. We propose a multimodal learning method that fuses roughly aligned CBCT and CT scans and investigate the effect on segmentation performance. For this experiment we use synthetically generated data containing real CT and synthetic CBCT volumes with corresponding voxel annotations. We show that this fusion setup improves segmentation performance in $18$ out of $20$ investigated setups.

Figures (4)

• Figure 1: Multimodal model configuration. After fusing the intraoperative CBCT and preoperative CT (early fusion), the data was processed by the 3D unet, segmenting liver and liver tumors. The figure further shows where late fusion would be applied.
• Figure 2: Sample CBCTLiTS data (subject 28): From left to right, the segmentation mask, CT scan, and various CBCT scans are shown, with CBCT quality degraded according to the undersampling factor $\alpha_{np}$.
• Figure 3: On the left, results of random affine augmentation ($\alpha_a = 0.25$) and the corresponding absolute difference image of $3$ different CBCTLiTS volumes are shown. The centered column shows the original data. Right, resulting volumes of random elastic misalignment ($\alpha_a = 0.25$) of the same volumes and the corresponding difference images are shown. Elastic misalignment is shown separate from affine misalignment for easier readability.
• Figure 4: Experimental results shown as boxplots. In the top row, results for liver segmentation are shown with Dice score on the y axis and the volume quality ($\alpha_{np}$) on the x axis. The median is shown as an orange bar and the mean value is printed in the corresponding box. Baseline results are displayed in blue and our proposed, multimodal method in green. The bottom row shows liver tumor segmentation. In the first column, experiments were performed on affine and the second column on combined affine and elastic transformed volumes.