Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Brain Imaging Generation with Latent Diffusion Models

Walter H. L. Pinaya, Petru-Daniel Tudosiu, Jessica Dafflon, Pedro F da Costa, Virginia Fernandez, Parashkev Nachev, Sebastien Ourselin, M. Jorge Cardoso

TL;DR

The paper addresses the data scarcity challenge in medical imaging by applying Latent Diffusion Models to generate high-resolution 3D brain MRI data. It leverages a latent autoencoder to scale diffusion to 3D volumes and implements conditioning on covariates such as age, sex, and brain structure volumes. The authors demonstrate high-quality synthetic samples, strong conditioning accuracy, and release a 100k-image dataset to the community, outperforming GAN baselines in unconditional generation. The work offers a practical path for data augmentation and privacy-preserving research in neuroimaging, with avenues for multi-modal conditioning in future work.

Abstract

Deep neural networks have brought remarkable breakthroughs in medical image analysis. However, due to their data-hungry nature, the modest dataset sizes in medical imaging projects might be hindering their full potential. Generating synthetic data provides a promising alternative, allowing to complement training datasets and conducting medical image research at a larger scale. Diffusion models recently have caught the attention of the computer vision community by producing photorealistic synthetic images. In this study, we explore using Latent Diffusion Models to generate synthetic images from high-resolution 3D brain images. We used T1w MRI images from the UK Biobank dataset (N=31,740) to train our models to learn about the probabilistic distribution of brain images, conditioned on covariables, such as age, sex, and brain structure volumes. We found that our models created realistic data, and we could use the conditioning variables to control the data generation effectively. Besides that, we created a synthetic dataset with 100,000 brain images and made it openly available to the scientific community.

Brain Imaging Generation with Latent Diffusion Models

TL;DR

The paper addresses the data scarcity challenge in medical imaging by applying Latent Diffusion Models to generate high-resolution 3D brain MRI data. It leverages a latent autoencoder to scale diffusion to 3D volumes and implements conditioning on covariates such as age, sex, and brain structure volumes. The authors demonstrate high-quality synthetic samples, strong conditioning accuracy, and release a 100k-image dataset to the community, outperforming GAN baselines in unconditional generation. The work offers a practical path for data augmentation and privacy-preserving research in neuroimaging, with avenues for multi-modal conditioning in future work.

Abstract

Deep neural networks have brought remarkable breakthroughs in medical image analysis. However, due to their data-hungry nature, the modest dataset sizes in medical imaging projects might be hindering their full potential. Generating synthetic data provides a promising alternative, allowing to complement training datasets and conducting medical image research at a larger scale. Diffusion models recently have caught the attention of the computer vision community by producing photorealistic synthetic images. In this study, we explore using Latent Diffusion Models to generate synthetic images from high-resolution 3D brain images. We used T1w MRI images from the UK Biobank dataset (N=31,740) to train our models to learn about the probabilistic distribution of brain images, conditioned on covariables, such as age, sex, and brain structure volumes. We found that our models created realistic data, and we could use the conditioning variables to control the data generation effectively. Besides that, we created a synthetic dataset with 100,000 brain images and made it openly available to the scientific community.
Paper Structure (10 sections, 4 figures, 1 table)

This paper contains 10 sections, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methods
  3. Datasets and Image Preprocessing
  4. Generative models
  5. Experiments
  6. Sampling Quality
  7. Conditioning Evaluation
  8. Synthetic Dataset
  9. Conclusions
  10. Acknowledgements

Figures (4)

  • Figure 1: Real and synthetic samples of head MRI generated using VAE-GAN, LSGAN, LDM and LDM+DDIM.
  • Figure 2: Conditioned sampling varying the ventricular volume and the brain volume normalized by the intracranial volume. In both rows, we kept the other variables constant.
  • Figure 3: Conditioning analysis. Left) Correlation between inputted ventricular volume vs ventricular measured with SynthSeg. Right) Correlation between inputted age and predicted brain age.
  • Figure 4: Extrapolating values of conditioning variables. During the training of the models, the inputted values of the conditioning variables were scaled between 0 and 1. In this experiment, we tried values outside of this range, and we observed that our model could extrapolate the representation of brain and ventricular volumes, showing that it learned the concept of these variables.