Simulating Dynamic Tumor Contrast Enhancement in Breast MRI using Conditional Generative Adversarial Networks

TL;DR

This work addresses the need for non-invasive breast cancer imaging by synthesizing dynamic DCE-MRI sequences from non-contrast MRI via a conditional GAN framework. It introduces the Scaled Aggregate Measure (SAMe) to unify multiple quality metrics and demonstrates both single-sequence and joint multi-timepoint DCE-MRI generation, evaluating their utility for tumor segmentation. Across experiments on the Duke dataset, synthetic post-contrast images closely resemble real post-contrast data and can improve segmentation performance, especially under domain shift, while joint timepoint generation captures clinically relevant contrast-kinetics patterns. The approach holds promise for CA-free screening and robust training data augmentation, with potential to enhance diagnostic workflows and patient safety in breast cancer management.

Abstract

This paper presents a method for virtual contrast enhancement in breast MRI, offering a promising non-invasive alternative to traditional contrast agent-based DCE-MRI acquisition. Using a conditional generative adversarial network, we predict DCE-MRI images, including jointly-generated sequences of multiple corresponding DCE-MRI timepoints, from non-contrast-enhanced MRIs, enabling tumor localization and characterization without the associated health risks. Furthermore, we qualitatively and quantitatively evaluate the synthetic DCE-MRI images, proposing a multi-metric Scaled Aggregate Measure (SAMe), assessing their utility in a tumor segmentation downstream task, and conclude with an analysis of the temporal patterns in multi-sequence DCE-MRI generation. Our approach demonstrates promising results in generating realistic and useful DCE-MRI sequences, highlighting the potential of virtual contrast enhancement for improving breast cancer diagnosis and treatment, particularly for patients where contrast agent administration is contraindicated.

Figures (8)

• Figure 1: Overview of pre- to post-contrast DCE-MRI synthesis using deep generative models, thereby localizing the contrast-enhanced tumor. Extending single sequence to multi-sequence DCE-MRI image generation further allows the characterization of tumors based on their temporal patterns of contrast agent uptake. The resulting synthetic images can be added as training data for downstream tasks (e.g., tumor segmentation), but, as shown, they can also be utilized to compute subtraction images commonly used in clinical settings for the diagnosis and treatment of breast cancer.
• Figure 2: Overview of training workflow of our pre- to post-contrast translating GAN based on Pix2PixHD wang2018high. Three reconstruction losses (L1) and two least squares adversarial losses (LSGAN)mao2017least from two discriminators (D1 & D2) and one pre-trained VGG simonyan2014very model are backpropagated into the generator, where lambda ($\lambda$) represents the weight of each of the different losses. Processing the images at two different scales inside the generator architecture balances local detail and global consistencywang2018high, which is further enforced by the two different image input scales in D1 (full size) and D2 (downsampled). The segmentation method is based on 3D U-Nets ronneberger2015u from the nnU-Net isensee2021nnu framework. The iteratively translated synthetic post-contrast axial slices are stacked to create 3D breast MRI volumes. These synthetic volumes correspond to the tumor segmentation masks, which were initially acquired based on the real post-contrast fat-saturated sequence.
• Figure 3: Depiction of the generally applicable dimensions of trustworthy synthetic data alongside respective examples (in blue font) for their adoption in deep generative models for medical image synthesis. The present study evaluates fidelity, diversity, condition adherence, and utility. Privacy and fairness are included herein for the comprehensiveness of the dimensions encompassing trustworthy synthetic data.
• Figure 4: Quantitative (a) and qualitative (b) illustrations of SAMe applied to DCE-MRI synthesis
• Figure 5: Synthesis of breast DCE-MRI as shown for six casessaha2018machine. Two cases were manually selected from the validation set (1 row: Case 228 , 2 row: Case 886), two manually selected from the test set (3 row: Case 378, 4 row: Case 907), and two randomly selected from the test set (5 row: Case 041, 6 row: Case 045). From left to right, we illustrate axial slices of the (a) real T1-weighted pre-contrast MRI, (b) the real DCE-MRI sequence 1, (c) the synthetic DCE-MRI sequence 1, the subtraction image based on the (d) real and (e) synthetic DCE-MRI subtractions, and (f) the ground truth segmentation mask.