Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Patient-Specific Autoregressive Models for Organ Motion Prediction in Radiotherapy

Yuxiang Lai, Jike Zhong, Vanessa Su, Xiaofeng Yang

TL;DR

This work addresses organ motion prediction in radiotherapy, where breathing-induced motion complicates precise dose delivery and traditional PCA-based DVF approaches falter due to registration dependence and lack of temporal dynamics. It introduces Auto-RMP, an autoregressive framework that treats 4D-CT phase sequences $X_t$ as a time-series and models the joint likelihood $P(X_{1},\dots,X_{T})=\prod_{t=1}^{T} p_\theta(X_t'|X_0,\dots,X_{t-1})$ to forecast multiple future CT phases from prior phases. The pipeline uses VQGAN-based CT tokenization to convert images into discrete tokens and a unidirectional Transformer (LLaMA-style) to generate future phase tokens, with organ masks produced by nnUNet serving as auxiliary inputs. Across public and private 4D-CT datasets totaling over $1{,}300$ phases, Auto-RMP achieves state-of-the-art IoU and DSC for lung and heart motion, and demonstrates robust long-term predictions by conditioning on the full input sequence and feeding its own predictions back into the input. This DVF-free, patient-specific autoregressive modeling approach has meaningful potential to improve pre-treatment planning and enable more adaptive radiotherapy.

Abstract

Radiotherapy often involves a prolonged treatment period. During this time, patients may experience organ motion due to breathing and other physiological factors. Predicting and modeling this motion before treatment is crucial for ensuring precise radiation delivery. However, existing pre-treatment organ motion prediction methods primarily rely on deformation analysis using principal component analysis (PCA), which is highly dependent on registration quality and struggles to capture periodic temporal dynamics for motion modeling.In this paper, we observe that organ motion prediction closely resembles an autoregressive process, a technique widely used in natural language processing (NLP). Autoregressive models predict the next token based on previous inputs, naturally aligning with our objective of predicting future organ motion phases. Building on this insight, we reformulate organ motion prediction as an autoregressive process to better capture patient-specific motion patterns. Specifically, we acquire 4D CT scans for each patient before treatment, with each sequence comprising multiple 3D CT phases. These phases are fed into the autoregressive model to predict future phases based on prior phase motion patterns. We evaluate our method on a real-world test set of 4D CT scans from 50 patients who underwent radiotherapy at our institution and a public dataset containing 4D CT scans from 20 patients (some with multiple scans), totaling over 1,300 3D CT phases. The performance in predicting the motion of the lung and heart surpasses existing benchmarks, demonstrating its effectiveness in capturing motion dynamics from CT images. These results highlight the potential of our method to improve pre-treatment planning in radiotherapy, enabling more precise and adaptive radiation delivery.

Patient-Specific Autoregressive Models for Organ Motion Prediction in Radiotherapy

TL;DR

This work addresses organ motion prediction in radiotherapy, where breathing-induced motion complicates precise dose delivery and traditional PCA-based DVF approaches falter due to registration dependence and lack of temporal dynamics. It introduces Auto-RMP, an autoregressive framework that treats 4D-CT phase sequences as a time-series and models the joint likelihood to forecast multiple future CT phases from prior phases. The pipeline uses VQGAN-based CT tokenization to convert images into discrete tokens and a unidirectional Transformer (LLaMA-style) to generate future phase tokens, with organ masks produced by nnUNet serving as auxiliary inputs. Across public and private 4D-CT datasets totaling over phases, Auto-RMP achieves state-of-the-art IoU and DSC for lung and heart motion, and demonstrates robust long-term predictions by conditioning on the full input sequence and feeding its own predictions back into the input. This DVF-free, patient-specific autoregressive modeling approach has meaningful potential to improve pre-treatment planning and enable more adaptive radiotherapy.

Abstract

Radiotherapy often involves a prolonged treatment period. During this time, patients may experience organ motion due to breathing and other physiological factors. Predicting and modeling this motion before treatment is crucial for ensuring precise radiation delivery. However, existing pre-treatment organ motion prediction methods primarily rely on deformation analysis using principal component analysis (PCA), which is highly dependent on registration quality and struggles to capture periodic temporal dynamics for motion modeling.In this paper, we observe that organ motion prediction closely resembles an autoregressive process, a technique widely used in natural language processing (NLP). Autoregressive models predict the next token based on previous inputs, naturally aligning with our objective of predicting future organ motion phases. Building on this insight, we reformulate organ motion prediction as an autoregressive process to better capture patient-specific motion patterns. Specifically, we acquire 4D CT scans for each patient before treatment, with each sequence comprising multiple 3D CT phases. These phases are fed into the autoregressive model to predict future phases based on prior phase motion patterns. We evaluate our method on a real-world test set of 4D CT scans from 50 patients who underwent radiotherapy at our institution and a public dataset containing 4D CT scans from 20 patients (some with multiple scans), totaling over 1,300 3D CT phases. The performance in predicting the motion of the lung and heart surpasses existing benchmarks, demonstrating its effectiveness in capturing motion dynamics from CT images. These results highlight the potential of our method to improve pre-treatment planning in radiotherapy, enabling more precise and adaptive radiation delivery.
Paper Structure (8 sections, 2 equations, 3 figures, 2 tables)

This paper contains 8 sections, 2 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Autoregressive Radiotherapy Motion Prediction
  3. Pipeline
  4. Implementation Details
  5. Experiment & Result
  6. Experiment
  7. Results
  8. Conclusion & Discussion

Figures (3)

  • Figure 1: Throughout the extended duration of radiotherapy, patients' natural breathing motion results in the continuous rhythmic expansion and contraction of the lungs. Consequently, the tumor moves up and down in the coronal view (dashed curve). However, current treatment plans often designate the target with a large margin for dose delivery (red rectangle), which accounts for tumor motion but results in unintended radiation doses to surrounding healthy tissues. By modeling and predicting organ motion patterns, dynamic radiation regions can be pre-defined, reducing target margins to enable more precise and adaptive radiation delivery.
  • Figure 2: Auto-RMP. We first arranges each phase of a 4D CT scan into an input image sequence ($X_{0}$ to $X_{T}$). Then, VQGAN serves as the vision encoder to convert CT images into discrete tokens. The output tokens are arranged into a 1D sequence, which is fed into the Autoregressive Transformer Prediction model. During the autoregressive process, the model predicts the next phase tokens based on the previous phase. Finally, the predicted phase tokens are decoded back into CT phases using the VQGAN decoder.
  • Figure 3: Long-term motion prediction. We evaluate the ability of models to perform long-term motion prediction. In this setting, the model is given the first five phases of the 4D CT scan and must predict the next five phases to assess its capability for learning long-term motion patterns. Auto-RMP demonstrates strong robustness in long-term motion prediction, generating smooth and consistent motion predictions.