Interactive Gadolinium-Free MRI Synthesis: A Transformer with Localization Prompt Learning

TL;DR

The paper tackles safety concerns associated with gadolinium-based contrast agents by proposing a gadolinium-free CE-MRI synthesis framework that uses a Restormer-based, multi-scale Transformer backbone augmented with Local and Global Fusion modules. A Location Prompt Module and a Fuzzy Prompt Generation mechanism enable ROI-guided synthesis during training, while interactive prompts can guide inference, bridging AI with radiologist expertise. Empirical results on the BraTS2021 dataset show superior PSNR and SSIM improvements over several cross-modality baselines, with notable gains in single-input scenarios and robust performance under multi-input conditions. The study advances interactive medical image synthesis, offering a practical path toward safer diagnostic procedures and potential downstream benefits in segmentation and detection tasks.

Abstract

Contrast-enhanced magnetic resonance imaging (CE-MRI) is crucial for tumor detection and diagnosis, but the use of gadolinium-based contrast agents (GBCAs) in clinical settings raises safety concerns due to potential health risks. To circumvent these issues while preserving diagnostic accuracy, we propose a novel Transformer with Localization Prompts (TLP) framework for synthesizing CE-MRI from non-contrast MR images. Our architecture introduces three key innovations: a hierarchical backbone that uses efficient Transformer to process multi-scale features; a multi-stage fusion system consisting of Local and Global Fusion modules that hierarchically integrate complementary information via spatial attention operations and cross-attention mechanisms, respectively; and a Fuzzy Prompt Generation (FPG) module that enhances the TLP model's generalization by emulating radiologists' manual annotation through stochastic feature perturbation. The framework uniquely enables interactive clinical integration by allowing radiologists to input diagnostic prompts during inference, synergizing artificial intelligence with medical expertise. This research establishes a new paradigm for contrast-free MRI synthesis while addressing critical clinical needs for safer diagnostic procedures. Codes are available at https://github.com/ChanghuiSu/TLP.

Figures (6)

• Figure 1: Structure diagram of the generator for dual-input tasks. Transformer modules are introduced at different levels to capture multi-scale information. Fusion modules and cross-fusion modules integrate the information from the two input modalities at local and global levels, respectively. The central spatial self-attention layer further enhances critical features. The prompt generation module operates only during the training phase, providing tumor location prompts to the model and enabling interactivity.
• Figure 2: Visual comparison of synthesis results from all one-to-one models on the BraTS 2021 dataset. Each model takes T1 as input simultaneously to synthesize T1ce.
• Figure 3: Visual comparison of synthesis results from all one-to-one models on the BraTS 2021 dataset. Each model takes T2 as inputs simultaneously to synthesize T1ce.
• Figure 4: Visual comparison of synthesis results from all many-to-one models on the BraTS 2021 dataset. Each model takes T1 and T2 as inputs simultaneously to synthesize T1ce.
• Figure 5: When the tumor shape is complex, the effect of the prompts is demonstrated.