A Biophysically-Conditioned Generative Framework for 3D Brain Tumor MRI Synthesis

TL;DR

The paper introduces a 3D latent diffusion framework conditioned on tissue segmentations and continuous tumor concentrations to synthesize anatomically consistent brain MRIs with tumors and to perform healthy tissue inpainting. It leverages a pretrained MAISI VAE for efficient latent representation and employs ControlNet-style conditioning, known-region injection, and RePaint-based refinements to ensure spatial coherence. Quantitative results on BraTS-based inpainting show competitive PSNR/SSIM metrics for both healthy and tumorous regions, with ablations illustrating the value of postprocessing. The approach offers a unified, biophysically informed pathway for realistic tumor synthesis and lesion removal, with potential for extension to other modalities and dynamic tumor modeling.

Abstract

Magnetic resonance imaging (MRI) inpainting supports numerous clinical and research applications. We introduce the first generative model that conditions on voxel-level, continuous tumor concentrations to synthesize high-fidelity brain tumor MRIs. For the BraTS 2025 Inpainting Challenge, we adapt this architecture to the complementary task of healthy tissue restoration by setting the tumor concentrations to zero. Our latent diffusion model conditioned on both tissue segmentations and the tumor concentrations generates 3D spatially coherent and anatomically consistent images for both tumor synthesis and healthy tissue inpainting. For healthy inpainting, we achieve a PSNR of 18.5, and for tumor inpainting, we achieve 17.4. Our code is available at: https://github.com/valentin-biller/ldm.git

Figures (11)

• Figure 1: For the training of our model, we input the tissue segmentations and tumor concentrations as conditions to the latent diffusion model. During inference, we set the tumor concentration to 0 to inpaint the voided regions as healthy brain tissue.
• Figure 1: Quantitative values for both healthy tissue (a) and tumor (b) inpainting.
• Figure 2: Qualitative inpainting results are presented for four representative subjects across the coronal, sagittal, and axial views. We showcase our ability to reconstruct missing regions in a 3D spatially coherent and anatomically consistent manner, both for tumor reconstruction and healthy tissue, with a single model.
• Figure 3: Violin plots of quantitative metrics for healthy (a) and tumor (b) inpainting. Median performance is indicated by a thick horizontal line, mean performance by a rhombus. The box bounds represent the first and third quartiles, and density is shown by the violin plot.
• Figure 4: Quantitative values for validation and test datasets of the BraTS Inpainting Challenge.