From Healthy Scans to Annotated Tumors: A Tumor Fabrication Framework for 3D Brain MRI Synthesis

TL;DR

The paper tackles the scarcity of annotated MRI tumor data by introducing Tumor Fabrication (TF), a two-stage framework that turns healthy 3D brain scans into realistic tumor-bearing images with paired labels using only a small set of real tumor data. TF-Aug generates coarse image–label pairs through ROI-based augmentation on healthy scans, and TF-GAN refines these into high-fidelity tumor-bearing volumes with a dual-head discriminator and class-wise perceptual guidance. Across BraTS 2023, TF data enrichment yields statistically significant segmentation gains in low-data regimes, outperforming CarveMix and Pix2Pix baselines and demonstrating robustness to varying synthetic data volumes. The results highlight the practical potential of leveraging abundant healthy data to enrich supervised learning in clinical AI, while acknowledging limitations in edema realism and mass-effect modeling and outlining directions for more realistic, customizable synthesis.

Abstract

The scarcity of annotated Magnetic Resonance Imaging (MRI) tumor data presents a major obstacle to accurate and automated tumor segmentation. While existing data synthesis methods offer promising solutions, they often suffer from key limitations: manual modeling is labor intensive and requires expert knowledge. Deep generative models may be used to augment data and annotation, but they typically demand large amounts of training pairs in the first place, which is impractical in data limited clinical settings. In this work, we propose Tumor Fabrication (TF), a novel two-stage framework for unpaired 3D brain tumor synthesis. The framework comprises a coarse tumor synthesis process followed by a refinement process powered by a generative model. TF is fully automated and leverages only healthy image scans along with a limited amount of real annotated data to synthesize large volumes of paired synthetic data for enriching downstream supervised segmentation training. We demonstrate that our synthetic image-label pairs used as data enrichment can significantly improve performance on downstream tumor segmentation tasks in low-data regimes, offering a scalable and reliable solution for medical image enrichment and addressing critical challenges in data scarcity for clinical AI applications.

Figures (3)

• Figure 1: Overview of the proposed framework. It consists of two stages: (1) TF-Aug: generation of coarse synthetic image–label pairs $(\mathbf{x}_{s'}, m_s)$ using ROI augmentations, and (2) refinement via TF-GAN, which includes a generator $G$, a dual-head discriminator $D$, and a frozen feature extractor $F$. The training is guided by region-specific losses: $\mathcal{L}_{n\text{-}roi}$ (hinge L1 loss for non-tumor regions) and $\mathcal{L}_{roi}$ (class-wise perceptual loss for tumor subregions).
• Figure 2: Sample visualization from TF-GAN refinement process. From left to right: Coarse synthetic image $\mathbf{x}_{s'}$, Synthetic mask $m_s$, ROI mask $m_{roi}$, Refined synthetic image $\mathbf{x}_s$ and an unpaired real image $\mathbf{x}_r$.
• Figure 3: Qualitative comparison between our method and other baselines. For each method, two representative samples are shown, including the synthetic image and its corresponding segmentation labels. For Tf-Aug and TF-GAN, identical input masks are used to enable direct visual comparison of refinement quality.