CT respiratory motion synthesis using joint supervised and adversarial learning

TL;DR

This work tackles the challenge of 4DCT-based motion assessment in radiotherapy by proposing a deep image synthesis framework that generates pseudo respiratory CT phases from a static 3D CT. It learns patient-specific deformation vector fields (DVFs) conditioned on external respiratory amplitude via an AdaIN-based scalar conditioning layer and trains with a dual loss that combines a supervised DVF reconstruction term and an adversarial term on both the warped image and the DVF magnitude. Validated on two diverse lung datasets, the method achieves motion accuracy comparable to repeat 4DCT scans, with tumor CoM errors around 2–3 mm and DSCs near 0.63–0.71 for synthetic phases, and substantial improvements in lung and organ motion metrics. The approach, which reduces dependence on multi-phase 4DCT during treatment planning, provides reproducible code and demonstrates practical potential for 4DCT-free motion-aware radiotherapy planning; future work includes more complex respiration conditioning and dosimetric impact assessment.

Abstract

Objective: Four-dimensional computed tomography (4DCT) imaging consists in reconstructing a CT acquisition into multiple phases to track internal organ and tumor motion. It is commonly used in radiotherapy treatment planning to establish planning target volumes. However, 4DCT increases protocol complexity, may not align with patient breathing during treatment, and lead to higher radiation delivery. Approach: In this study, we propose a deep synthesis method to generate pseudo respiratory CT phases from static images for motion-aware treatment planning. The model produces patient-specific deformation vector fields (DVFs) by conditioning synthesis on external patient surface-based estimation, mimicking respiratory monitoring devices. A key methodological contribution is to encourage DVF realism through supervised DVF training while using an adversarial term jointly not only on the warped image but also on the magnitude of the DVF itself. This way, we avoid excessive smoothness typically obtained through deep unsupervised learning, and encourage correlations with the respiratory amplitude. Main results: Performance is evaluated using real 4DCT acquisitions with smaller tumor volumes than previously reported. Results demonstrate for the first time that the generated pseudo-respiratory CT phases can capture organ and tumor motion with similar accuracy to repeated 4DCT scans of the same patient. Mean inter-scans tumor center-of-mass distances and Dice similarity coefficients were $1.97$mm and $0.63$, respectively, for real 4DCT phases and $2.35$mm and $0.71$ for synthetic phases, and compares favorably to a state-of-the-art technique (RMSim).

Figures (8)

• Figure 1: Variability of 4DCT tumor tracking. Here, two patients underwent multiple repeat 4DCT scans due to excess variability between corresponding phases. Green, yellow, and red dots indicate the center of the tumor in scans 1, 2, and 3 respectively. Colored crosses show the corresponding tumor position in the other scans. Images from CHRU Brest.
• Figure 2: Comparison of typical appearances of deformation vector fields obtained from state-of-the art (a) conventional lung DIR vishnevskiy_isotropic_2017 and (b) deep learning-based lung DIR hansen_graphregnet_2021.
• Figure 3: Proposed convolutional DVF generating architecture with scalar conditioning. The respiratory amplitude is injected through an AdaIN mechanism at the bottleneck of the encoder. Given an input image $X$ and a respiratory amplitude $\alpha$, the model learns a DVF $\phi^*_\alpha$ from supervised labels $\phi_\alpha$. Then the input image is warped with a spatial transformer to obtain the synthetic phase $Y^* = X\circ \phi^*_\alpha$. An adversarial loss based on the concatenation of the warped image and the magnitude of the DVF $||\phi_\alpha||$ is used to improve the realism of the DVF without explicitly constraining its smoothness.
• Figure 4: Characteristics of the two 4DCT datasets used in our experiments.
• Figure 5: Influence of the loss terms on the generated DVF. (a) Reconstruction loss only: $\ell_1$ on the DVF (b) Addition of an adversarial term on the warped image (c) Proposed compound loss: $\ell_1$ on the DVF + adversarial term jointly on the warped image and the DVF magnitude. (d) Reference state-of-the-art DVF obtained by vishnevskiy_isotropic_2017