Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MAM-E: Mammographic synthetic image generation with diffusion models

Ricardo Montoya-del-Angel, Karla Sam-Millan, Joan C Vilanova, Robert Martí

TL;DR

The paper tackles data scarcity in mammography by introducing MAM-E, a diffusion-model pipeline that generates high-quality healthy mammograms conditioned on text prompts and performs lesion inpainting in specified regions using latent stable diffusion and DreamBooth fine-tuning. It leverages two datasets (OMI-H and VinDr-Mammo) to train separate and combined models, enabling cross-dataset concept extrapolation and region-specific lesion synthesis, all accessible via a graphical user interface. Evaluation includes radiological assessment showing realism close to real images (AUROC near 0.5 for distinguishing synthetic from real) and preliminary CAD/XAI analyses suggesting synthetic lesions are visually and distributionally plausible, with limitations in resolution and pixel depth. The work demonstrates a feasible path to augment mammography datasets with controllable, image-conditioned synthetic data and provides open-source weights and interfaces for broader adoption and further CAD validation.

Abstract

Generative models are used as an alternative data augmentation technique to alleviate the data scarcity problem faced in the medical imaging field. Diffusion models have gathered special attention due to their innovative generation approach, the high quality of the generated images and their relatively less complex training process compared with Generative Adversarial Networks. Still, the implementation of such models in the medical domain remains at early stages. In this work, we propose exploring the use of diffusion models for the generation of high quality full-field digital mammograms using state-of-the-art conditional diffusion pipelines. Additionally, we propose using stable diffusion models for the inpainting of synthetic lesions on healthy mammograms. We introduce MAM-E, a pipeline of generative models for high quality mammography synthesis controlled by a text prompt and capable of generating synthetic lesions on specific regions of the breast. Finally, we provide quantitative and qualitative assessment of the generated images and easy-to-use graphical user interfaces for mammography synthesis.

MAM-E: Mammographic synthetic image generation with diffusion models

TL;DR

The paper tackles data scarcity in mammography by introducing MAM-E, a diffusion-model pipeline that generates high-quality healthy mammograms conditioned on text prompts and performs lesion inpainting in specified regions using latent stable diffusion and DreamBooth fine-tuning. It leverages two datasets (OMI-H and VinDr-Mammo) to train separate and combined models, enabling cross-dataset concept extrapolation and region-specific lesion synthesis, all accessible via a graphical user interface. Evaluation includes radiological assessment showing realism close to real images (AUROC near 0.5 for distinguishing synthetic from real) and preliminary CAD/XAI analyses suggesting synthetic lesions are visually and distributionally plausible, with limitations in resolution and pixel depth. The work demonstrates a feasible path to augment mammography datasets with controllable, image-conditioned synthetic data and provides open-source weights and interfaces for broader adoption and further CAD validation.

Abstract

Generative models are used as an alternative data augmentation technique to alleviate the data scarcity problem faced in the medical imaging field. Diffusion models have gathered special attention due to their innovative generation approach, the high quality of the generated images and their relatively less complex training process compared with Generative Adversarial Networks. Still, the implementation of such models in the medical domain remains at early stages. In this work, we propose exploring the use of diffusion models for the generation of high quality full-field digital mammograms using state-of-the-art conditional diffusion pipelines. Additionally, we propose using stable diffusion models for the inpainting of synthetic lesions on healthy mammograms. We introduce MAM-E, a pipeline of generative models for high quality mammography synthesis controlled by a text prompt and capable of generating synthetic lesions on specific regions of the breast. Finally, we provide quantitative and qualitative assessment of the generated images and easy-to-use graphical user interfaces for mammography synthesis.
Paper Structure (19 sections, 6 figures, 2 tables)

This paper contains 19 sections, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Datasets
  4. OMI-H
  5. VinDr-Mammo
  6. Data preprocessing and preparation
  7. Healthy image generation
  8. Lesion inpainting
  9. Diffusion models
  10. Healthy image generation
  11. Lesion inpainting
  12. Experiments
  13. Healthy mammogram generation
  14. Independent datasets
  15. Concept extrapolation
  16. ...and 4 more sections

Figures (6)

  • Figure 1: Graphical user interface of MAM-E for generation of synthetic healthy mammograms
  • Figure 2: Inpainting training pipeline. The mask is reshaped to match the image size of the latent representations (64x64). The same UNet as in the SD pipeline is used.
  • Figure 3: Training evolution of SDM with Hologic images at epoch 1, 3, 6 and 10. The prompt is: "a mammogram in MLO view with small area".
  • Figure 4: Training evolution of the diffusion process on a conditional pretrained model trained with both Siemens and Hologic images at epoch 1, 3, 7 and 40. The prompt is: "a siemens mammogram in MLO view with high density and small area".
  • Figure 5: MAM-E lesion drawing tool.
  • ...and 1 more figures