HiFi-Syn: Hierarchical Granularity Discrimination for High-Fidelity Synthesis of MR Images with Structure Preservation

TL;DR

The model excels not only in synthesizing normal structures but also in handling abnormal (pathological) structures, such as brain tumors, despite the variations in contrast observed across different imaging modalities due to their pathological characteristics.

Abstract

Synthesizing medical images while preserving their structural information is crucial in medical research. In such scenarios, the preservation of anatomical content becomes especially important. Although recent advances have been made by incorporating instance-level information to guide translation, these methods overlook the spatial coherence of structural-level representation and the anatomical invariance of content during translation. To address these issues, we introduce hierarchical granularity discrimination, which exploits various levels of semantic information present in medical images. Our strategy utilizes three levels of discrimination granularity: pixel-level discrimination using a Brain Memory Bank, structure-level discrimination on each brain structure with a re-weighting strategy to focus on hard samples, and global-level discrimination to ensure anatomical consistency during translation. The image translation performance of our strategy has been evaluated on three independent datasets (UK Biobank, IXI, and BraTS 2018), and it has outperformed state-of-the-art algorithms. Particularly, our model excels not only in synthesizing normal structures but also in handling abnormal (pathological) structures, such as brain tumors, despite the variations in contrast observed across different imaging modalities due to their pathological characteristics. The diagnostic value of synthesized MR images containing brain tumors has been evaluated by radiologists. This indicates that our model may offer an alternative solution in scenarios where specific MR modalities of patients are unavailable. Extensive experiments further demonstrate the versatility of our method, providing unique insights into medical image translation.

Figures (9)

• Figure 1: Illustration of our motivation. Current Image-to-Image (I2I) methods, both (a) global-level and (b) instance-level, often overlook the spatial coherence of structural-level representation and the anatomical consistency of content during translation. To address this limitation, we introduce a (c) structure-level content translation method for synthesizing MR images containing both normal and abnormal tissues.
• Figure 2: The overview of our proposed framework. The input images are encoded to content and attribute features and then processed by Pixel-level Granularity Discrimination (PGD) module, Structure-level Granularity Discrimination (SGD) module, and Global-level Granularity Discrimination (GGD) module simultaneously to achieve structure-preserving translation. Note that the annotations are unseen during testing. For the sake of conciseness, the second-stage style translation is not illustrated here.
• Figure 3: Illustration of memory item setting for the IXI/UKB datasets. Each query reads and updates the corresponding structure-aware items with labels and global items without labels during training. Items store domain-specific attribute representations, utilizing shared keys to access them.
• Figure 4: Comparison of I2I translation methods on the IXI and UKB datasets. (From the first to the last sample) T1$\rightarrow$T2, T2$\rightarrow$T1, T1$\rightarrow$T2-Flair, and T2-Flair$\rightarrow$T1 results. Compared to other SOTA methods, HiFi-Syn shows superior structure-preserving translation with clearer error maps.
• Figure 5: Comparison of I2I translation methods on the BraTS dataset. (From the first to the last sample) T1$\rightarrow$T2-Flair, T1$\rightarrow$T2, and T1ce$\rightarrow$T1 results. Compared to other SOTA methods, HiFi-Syn shows superior structure-preserving translation on both normal and non-normal MR images with clearer error maps. Note that annotations are unseen during testing.