Leveraging Multimodal CycleGAN for the Generation of Anatomically Accurate Synthetic CT Scans from MRIs

TL;DR

The paper tackles the challenge of generating anatomically accurate CT scans from MRIs to streamline MRI-based radiotherapy planning. It interrogates unpaired image-to-image translation via CycleGAN architectures, exploring single-input and multimodal MRI configurations to synthesize CT images across both contrast-enhanced and non-contrast modalities. The study employs distribution-based metrics (FID, KL divergence, histogram analysis) and spectral analysis, complemented by qualitative blind physician assessments, to evaluate performance in the absence of ground-truth CT–MRI pairs. Findings indicate multimodal inputs, particularly In-n-out HUM, produce CT-like distributions closest to real data, though 2D slice-wise generation and regional variability remain challenges, underscoring potential for data augmentation and workflow efficiency with further advances.

Abstract

In many clinical settings, the use of both Computed Tomography (CT) and Magnetic Resonance (MRI) is necessary to pursue a thorough understanding of the patient's anatomy and to plan a suitable therapeutical strategy; this is often the case in MRI-based radiotherapy, where CT is always necessary to prepare the dose delivery, as it provides the essential information about the radiation absorption properties of the tissues. Sometimes, MRI is preferred to contour the target volumes. However, this approach is often not the most efficient, as it is more expensive, time-consuming and, most importantly, stressful for the patients. To overcome this issue, in this work, we analyse the capabilities of different configurations of Deep Learning models to generate synthetic CT scans from MRI, leveraging the power of Generative Adversarial Networks (GANs) and, in particular, the CycleGAN architecture, capable of working in an unsupervised manner and without paired images, which were not available. Several CycleGAN models were trained unsupervised to generate CT scans from different MRI modalities with and without contrast agents. To overcome the problem of not having a ground truth, distribution-based metrics were used to assess the model's performance quantitatively, together with a qualitative evaluation where physicians were asked to differentiate between real and synthetic images to understand how realistic the generated images were. The results show how, depending on the input modalities, the models can have very different performances; however, models with the best quantitative results, according to the distribution-based metrics used, can generate very difficult images to distinguish from the real ones, even for physicians, demonstrating the approach's potential.

Figures (10)

• Figure 1: Base scheme of a GAN. hitawala2018comparative
• Figure 2: Schematic representation of the CNN used in this work.
• Figure 3: An example of a portion of the survey proposed to the doctors. The language is Italian, as the participants were all Italians. Question 2. translates to: "Do you think the image is : -real; -synthetic; -indeterminable". Question 3. translates to: "If possible, briefly state the motivation for your answer".
• Figure 4: Representative slices for each of the ten portions in which the region is split for the analyses. The splits go from the lower regions encompassing the lowest portion of the kidneys and the bowels to the lowest portion of the lungs in layers 8, 9, and 10.
• Figure 5: Layer-wise FID scores for each model.