MRI to PET Cross-Modality Translation using Globally and Locally Aware GAN (GLA-GAN) for Multi-Modal Diagnosis of Alzheimer's Disease

TL;DR

This work tackles cross-modality MRI-to-PET translation to aid Alzheimer's disease diagnosis by introducing GLA-GAN, a globally and locally aware multi-path GAN that combines a global generator with patch-based local generators. The model employs a composite loss (adversarial, voxel-wise L1, MS-SSIM, ROI) to preserve global structure, local detail, and region-specific information, achieving superior PET synthesis quality compared with state-of-the-art methods. Quantitative results show improved SSIM and reduced MAE, and the synthesized PET scans also enhance AD classification performance when used for data augmentation or completion of incomplete datasets. The authors further explore interpretability through visualization of internal features and interpolation analyses, while acknowledging limitations such as increased parameter count and the need for broader validation across disease stages like MCI. Overall, the approach offers a promising path toward practical, multi-modal diagnostic pipelines with reduced PET acquisition needs.

Abstract

Medical imaging datasets are inherently high dimensional with large variability and low sample sizes that limit the effectiveness of deep learning algorithms. Recently, generative adversarial networks (GANs) with the ability to synthesize realist images have shown great potential as an alternative to standard data augmentation techniques. Our work focuses on cross-modality synthesis of fluorodeoxyglucose~(FDG) Positron Emission Tomography~(PET) scans from structural Magnetic Resonance~(MR) images using generative models to facilitate multi-modal diagnosis of Alzheimer's disease (AD). Specifically, we propose a novel end-to-end, globally and locally aware image-to-image translation GAN (GLA-GAN) with a multi-path architecture that enforces both global structural integrity and fidelity to local details. We further supplement the standard adversarial loss with voxel-level intensity, multi-scale structural similarity (MS-SSIM) and region-of-interest (ROI) based loss components that reduce reconstruction error, enforce structural consistency at different scales and perceive variation in regional sensitivity to AD respectively. Experimental results demonstrate that our GLA-GAN not only generates synthesized FDG-PET scans with enhanced image quality but also superior clinical utility in improving AD diagnosis compared to state-of-the-art models. Finally, we attempt to interpret some of the internal units of the GAN that are closely related to this specific cross-modality generation task.

Figures (8)

• Figure 1: Proposed globally and locally aware image-to-image translation GAN (GLA-GAN) architecture for cross-modal PET estimation from MRI using $L_1$, MS-SSIM, and ROI-based objective function. [Best viewed in color]
• Figure 2: Classification Model: Using features extracted from MR images and their corresponding synthetic PET images produced through various generative methods [Best viewed in color]
• Figure 3: Qualitative comparison of PET scans for Alzheimer’s disease (AD) sample: synthesized using GLA-GAN, Global-GAN, Local-GAN, 3D U-Net, Cycle GAN, and GAN. Ground truth, estimated PET scan, and error maps corresponding to each model are presented in axial, coronal, and sagittal viewse. [Best viewed in color]
• Figure 4: Qualitative comparison of PET scans for Control (CN) sample: Synthesized using GLA-GAN, Global-GAN, Local-GAN, 3D U-Net, Cycle GAN, and GAN. Ground truth, estimated PET scan, and error maps corresponding to each model are presented in axial, coronal, and sagittal views. [Best viewed in color]
• Figure 5: Evaluation of the GLA-GAN model with different combinations of loss components across standard classification performance metrics on both paired and complete datasets.