Bidirectional Brain Image Translation using Transfer Learning from Generic Pre-trained Models
Fatima Haimour, Rizik Al-Sayyed, Waleed Mahafza, Omar S. Al-Kadi
TL;DR
This work investigates bidirectional MR–CT brain image translation by transferring knowledge from 18 generic pre-trained models within a CycleGAN framework. Using a real paired MR–CT brain dataset and multiple image-quality metrics (PSNR, SSIM, UQI, VIF) plus radiologist perceptual validation, it demonstrates that certain non-medical pre-trained backbones can yield competitive translations, with iphone2dslr_flower consistently achieving top performance. Latent-space analyses reveal that successful models sharply separate MR and CT representations, reinforcing the need for domain-aligned pre-training. The study highlights transfer learning as a data-efficient approach for medical image synthesis and provides practical guidance on model selection and fine-tuning for brain imaging tasks toward potential clinical applications.
Abstract
Brain imaging plays a crucial role in the diagnosis and treatment of various neurological disorders, providing valuable insights into the structure and function of the brain. Techniques such as magnetic resonance imaging (MRI) and computed tomography (CT) enable non-invasive visualization of the brain, aiding in the understanding of brain anatomy, abnormalities, and functional connectivity. However, cost and radiation dose may limit the acquisition of specific image modalities, so medical image synthesis can be used to generate required medical images without actual addition. In the medical domain, where obtaining labeled medical images is labor-intensive and expensive, addressing data scarcity is a major challenge. Recent studies propose using transfer learning to overcome this issue. This involves adapting pre-trained CycleGAN models, initially trained on non-medical data, to generate realistic medical images. In this work, transfer learning was applied to the task of MR-CT image translation and vice versa using 18 pre-trained non-medical models, and the models were fine-tuned to have the best result. The models' performance was evaluated using four widely used image quality metrics: Peak-signal-to-noise-ratio, Structural Similarity Index, Universal Quality Index, and Visual Information Fidelity. Quantitative evaluation and qualitative perceptual analysis by radiologists demonstrate the potential of transfer learning in medical imaging and the effectiveness of the generic pre-trained model. The results provide compelling evidence of the model's exceptional performance, which can be attributed to the high quality and similarity of the training images to actual human brain images. These results underscore the significance of carefully selecting appropriate and representative training images to optimize performance in brain image analysis tasks.