AI-Based Detection of In-Treatment Changes from Prostate MR-Linac Images

TL;DR

This study demonstrates that AI can detect treatment-induced changes in longitudinal MR-Linac prostate images by predicting the correct temporal order of fraction pairs. A Siamese 3D CNN within the LILAC framework achieves near-perfect discrimination ($AUC\approx0.99$) on post-treatment pairs and outperforms expert readers, with performance scaling with inter-fraction interval. Saliency and input-ablation analyses localize changes mainly to the prostate, bladder, and pubic symphysis, suggesting RT-induced anatomical and textural alterations; quantitative findings corroborate prostate enlargement and bladder contraction with darker, more heterogeneous prostate signals. These results indicate MR-Linac imaging, coupled with AI, can serve as a biomarker for in-treatment changes and potentially support adaptive radiotherapy and toxicity prediction.

Abstract

Purpose: To investigate whether routinely acquired longitudinal MR-Linac images can be leveraged to characterize treatment-induced changes during radiotherapy, particularly subtle inter-fraction changes over short intervals (average of 2 days). Materials and Methods: This retrospective study included a series of 0.35T MR-Linac images from 761 patients. An artificial intelligence (deep learning) model was used to characterize treatment-induced changes by predicting the temporal order of paired images. The model was first trained with the images from the first and the last fractions (F1-FL), then with all pairs (All-pairs). Model performance was assessed using quantitative metrics (accuracy and AUC), compared to a radiologist's performance, and qualitative analyses - the saliency map evaluation to investigate affected anatomical regions. Input ablation experiments were performed to identify the anatomical regions altered by radiotherapy. The radiologist conducted an additional task on partial images reconstructed by saliency map regions, reporting observations as well. Quantitative image analysis was conducted to investigate the results from the model and the radiologist. Results: The F1-FL model yielded near-perfect performance (AUC of 0.99), significantly outperforming the radiologist. The All-pairs model yielded an AUC of 0.97. This performance reflects therapy-induced changes, supported by the performance correlation to fraction intervals, ablation tests and expert's interpretation. Primary regions driving the predictions were prostate, bladder, and pubic symphysis. Conclusion: The model accurately predicts temporal order of MR-Linac fractions and detects radiation-induced changes over one or a few days, including prostate and adjacent organ alterations confirmed by experts. This underscores MR-Linac's potential for advanced image analysis beyond image guidance.

Figures (9)

• Figure 1: Overview of the MR-Linac workflow and AI framework.
• Figure 2: (Continued) (A) Process of MR-Linac sessions across multiple fractions. Sim refers to the simulation scans obtained prior to MR-Linac treatment sessions. $F_1$, $F_2$, ..., $F_L$ represent MR images acquired at the first, second, ..., and last fractions, respectively, prior to image-guided radiation therapy (RT). (B) Schematic diagram of the LILAC-based AI model. Paired 3D MR image volumes from fractions are input to a Siamese 3D convolutional neural network with ResNet-18 He_2016_CVPR architecture, in which features of each image are obtained. The differences between these two features are fed into a fully connected layer. The logits are used for binary classification to predict if the image pairs are in the correct temporal order or not. (C) Experimental design for evaluation. A radiologist performed the same temporal-ordering task first with full volumes and next with saliency-restricted images. The radiologist also evaluated anatomical regions highlighted by the saliency maps. Model input ablation experiments were designed to evaluate altered anatomical regions. Performance changes when the prostate, bladder, or both regions or the surrounding box regions were masked in MR images (Organ Masked, Box Masked) were evaluated, as well as performance for partial MR images encompassing the organ regions (Only Organ). Performance for using only masks (Mask) was evaluated to investigate the effects of the organ shapes, compared to MR intensities.
• Figure 3: Pairwise performance (A) Model logits of the All-pairs model for different image pairs. Model logits reflect the model's confidence in its predictions, which can be interpreted as the magnitude of changes detected between paired images. Pairs of numbers on the x-axis indicate the fractions at which the paired images were acquired. Cases were grouped by the first fraction and ordered and color-coded by the interval between the paired fractions. Stars indicate statistical significance between compared cases (independent two-sided t-test; ns: $.05 < p$, **: $.001 < p \le .01$, ***: $.0001 < p \le .001$, ****: $p \le .0001$). Means within the same first fraction group are shown in red, whereas means with the same fraction interval but different first fraction groups are shown in green. (B) Linear mixed-effects model showing the relationship between fraction intervals from the first fraction and model logits (black), with patient-wise gradients shown in multicolor.
• Figure 4: Saliency map on an atlas. The atlas was built from $F_1$ of all patients in the test data. The visualized heatmap is the mean of saliency maps for all patients in the test data, obtained from All-pairs model inference on $F_1$-$F_L$ pairs and transformed into the atlas space. Column titles indicate the slice orientations (axial, sagittal) and the primary regions of interest highlighted by the saliency maps, although these regions span multiple organs.
• Figure 5: Changes in volume and intensity of prostate and bladder in MR images. An increase in the difference between the two means is indicated in red, whereas a decrease is indicated in blue. Stars indicate statistical significance of the difference (Wilcoxon signed-rank test; ****: $p \le .0001$)