HAGAN: Hybrid Augmented Generative Adversarial Network for Medical Image Synthesis

TL;DR

The paper tackles the challenge of preserving pathological integrity and local tissue detail in medical image synthesis. It proposes HAGAN, a hybrid framework combining AttnMix consistency differentiable regularization, a bi-level Hierarchical Discriminator, and a Reverse Skip Connection to enforce global and local fidelity. Through a bi-level loss design and differentiable augmentation, HAGAN achieves state-of-the-art FID scores on COVID-CT, ACDC, and BraTS2018 across 64×64 and 256×256 resolutions, with ablation studies showing progressive gains from each component. The approach offers scalable, efficient MIS generation with strong local detail and structural fidelity, suitable for data augmentation and clinical modeling. These results indicate substantial potential for improved MIS-based training and diagnosis in diverse clinical imaging scenarios.

Abstract

Medical Image Synthesis (MIS) plays an important role in the intelligent medical field, which greatly saves the economic and time costs of medical diagnosis. However, due to the complexity of medical images and similar characteristics of different tissue cells, existing methods face great challenges in meeting their biological consistency. To this end, we propose the Hybrid Augmented Generative Adversarial Network (HAGAN) to maintain the authenticity of structural texture and tissue cells. HAGAN contains Attention Mixed (AttnMix) Generator, Hierarchical Discriminator and Reverse Skip Connection between Discriminator and Generator. The AttnMix consistency differentiable regularization encourages the perception in structural and textural variations between real and fake images, which improves the pathological integrity of synthetic images and the accuracy of features in local areas. The Hierarchical Discriminator introduces pixel-by-pixel discriminant feedback to generator for enhancing the saliency and discriminance of global and local details simultaneously. The Reverse Skip Connection further improves the accuracy for fine details by fusing real and synthetic distribution features. Our experimental evaluations on three datasets of different scales, i.e., COVID-CT, ACDC and BraTS2018, demonstrate that HAGAN outperforms the existing methods and achieves state-of-the-art performance in both high-resolution and low-resolution.

Figures (7)

• Figure 1: Illustration of the real medical imaging samples in lungs and brain. The structural texture features of medical images are very complex and subtle. Moreover, the characteristics of tissue cells in medical imaging are similar. It is challenging to distinguish the feature disparities between key lesions and different tissues.
• Figure 2: Overview of proposed Hybrid Augmented Generative Adversarial Network(HAGAN). Given a random noise z, first synthesize the medical image through the generator, and use the AttnMix to complete the differentiable augmentation and mixing of real and fake images to ensure model to be perceptive in the structural and textural variations. After that, the T(fake), T(real) and Mix input to Hierarchical Discriminator, and calculates the bi-level adversarial loss and consistency loss to achieve joint learning to maintain structural integrity of pathology and consistency of local texture details.
• Figure 3: Illustration of AttnMix consistency differentiable regularization. P1 and P2 are the discrimination results of fake images and real images respectively.
• Figure 4: The structure of Reserve Skip Connection. The left is generator and the right is discriminator.
• Figure 5: Medical image synthesis samples on 64 × 64 resolution.