Pre- to Post-Contrast Breast MRI Synthesis for Enhanced Tumour Segmentation

TL;DR

This study tackles the risk and drawbacks of gadolinium-based contrast in breast MRI by generating synthetic post-contrast images from pre-contrast scans using a Pix2PixHD-based GAN. It introduces SAMe, a scaled ensemble of image-quality metrics ($SSIM$, $MSE$, $MAE$, $FID_{Img}$, $FID_{Rad}$) to consistently compare synthetic data and select training checkpoints. Quantitative and qualitative results show that synthetic post-contrast data approximate real post-contrast distributions and can significantly improve 3D breast tumour segmentation, especially under domain-shift scenarios where real post-contrast data are scarce. The work demonstrates the potential for reducing contrast administration while enhancing segmentation robustness, offering practical pathways for data augmentation and clinical risk mitigation.

Abstract

Despite its benefits for tumour detection and treatment, the administration of contrast agents in dynamic contrast-enhanced MRI (DCE-MRI) is associated with a range of issues, including their invasiveness, bioaccumulation, and a risk of nephrogenic systemic fibrosis. This study explores the feasibility of producing synthetic contrast enhancements by translating pre-contrast T1-weighted fat-saturated breast MRI to their corresponding first DCE-MRI sequence leveraging the capabilities of a generative adversarial network (GAN). Additionally, we introduce a Scaled Aggregate Measure (SAMe) designed for quantitatively evaluating the quality of synthetic data in a principled manner and serving as a basis for selecting the optimal generative model. We assess the generated DCE-MRI data using quantitative image quality metrics and apply them to the downstream task of 3D breast tumour segmentation. Our results highlight the potential of post-contrast DCE-MRI synthesis in enhancing the robustness of breast tumour segmentation models via data augmentation. Our code is available at https://github.com/RichardObi/pre_post_synthesis.

Figures (5)

• Figure 1: Overview of training workflow of our pre- to post-contrast translating Generative Adversarial Network (GAN) based on Pix2PixHD wang2018high. Three reconstruction losses (L1) and two least squares adversarial losses (LSGAN)mao2017least from two discriminators (D1 & D2) and one pretrained VGG simonyan2014very model are backpropagated into the generator, where lambda ($\lambda$) represents the weight of each of the different losses. Processing the images at two different scales inside the generator architecture balances local detail and global consistency, which is further enforced by the two different image input scales in D1 (full size) and D2 (downsampled).
• Figure 2: Overview of the segmentation method based on 3D U-Nets ronneberger2015u from the nnU-Net isensee2021nnu framework. The iteratively translated synthetic post-contrast axial slices are stacked to create 3D breast MRI volumes. These synthetic volumes correspond to the tumour segmentation masks, which were initially acquired based on the real post-contrast fat-saturated sequence.
• Figure 3: Breast DCE-MRI synthesis, as shown for six cases from the Duke Dataset saha2018machine, two of which are manually selected from the validation set (1-3 row: Case 228, 4 row: Case 886), two manually selected from the test set (5 row: Case 378, 6 row: Case 907), and two randomly selected from the test set (7 row: Case 041, 8 row: Case 045). From the left to the right, the corresponding axial slices are depicted for (a) the real pre-contrast, (b) the real post-contrast phase 1, (c) the synthetic post-contrast phase 1, (d) the subtraction image based on the real post-contrast image, (e) the subtraction image based on the synthethic post-contrast image, and (f) the ground truth segmentation mask. Intensity and contrast of the subtraction images was increased using OpenCVopencv (same scaling for all images). Samples of case 228 are shown in the axial (1 row, slice 111), saggital (2 row, slice 119), and coronal view (3 row, slice 286). The synthetic images from the coronal and sagittal planes are extracted from the respective 3D volume that is based on stacked synthetic 2D axial slices.
• Figure 4: Synthetic image distribution (FID$_{Img}$ and FID$_{Rad}$), and pixel space objective (MSE and MAE) and perception-based (SSIM) quality metrics across the generative models' training epochs. Utilising these, we introduce the Scaled Aggregate Measure (SAMe) to inspect the overall quality (the lower, the better) and enabling an informed selection of the best training checkpoint (i.e., epoch 30, achieving the lowest SAMe) for image generation. FID metrics are computed for 3000 and MSE, MAE and SSIM metrics for 5000 synthetic-real post-contrast pairs of axial MRI slices from the validation set.
• Figure 5: Single breast examples of cropped T1 MRI slices with tumour bounding box for two randomly selected test cases from the Duke Dataset saha2018machine. Case 612 (normal) is shown in the top row and Case 005 (difficult) in the bottom row. Each row is organised in 3 by 3 columns (order: axial, sagittal, coronal), where the first, second, and third column corresponds to pre-contrast, real post-contrast, and synthetic post-contrast, respectively. The intensity of these images was auto-adjusted using ITKSnap yushkevich2016itk.