Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Physics-Informed Latent Diffusion for Multimodal Brain MRI Synthesis

Sven Lüpke, Yousef Yeganeh, Ehsan Adeli, Nassir Navab, Azade Farshad

TL;DR

This work presents a novel physics-informed generative model capable of synthesizing a variable number of brain MRI modalities, including those not present in the original dataset, including those not present in the original dataset.

Abstract

Recent advances in generative models for medical imaging have shown promise in representing multiple modalities. However, the variability in modality availability across datasets limits the general applicability of the synthetic data they produce. To address this, we present a novel physics-informed generative model capable of synthesizing a variable number of brain MRI modalities, including those not present in the original dataset. Our approach utilizes latent diffusion models and a two-step generative process: first, unobserved physical tissue property maps are synthesized using a latent diffusion model, and then these maps are combined with a physical signal model to generate the final MRI scan. Our experiments demonstrate the efficacy of this approach in generating unseen MR contrasts and preserving physical plausibility. Furthermore, we validate the distributions of generated tissue properties by comparing them to those measured in real brain tissue.

Physics-Informed Latent Diffusion for Multimodal Brain MRI Synthesis

TL;DR

This work presents a novel physics-informed generative model capable of synthesizing a variable number of brain MRI modalities, including those not present in the original dataset, including those not present in the original dataset.

Abstract

Recent advances in generative models for medical imaging have shown promise in representing multiple modalities. However, the variability in modality availability across datasets limits the general applicability of the synthetic data they produce. To address this, we present a novel physics-informed generative model capable of synthesizing a variable number of brain MRI modalities, including those not present in the original dataset. Our approach utilizes latent diffusion models and a two-step generative process: first, unobserved physical tissue property maps are synthesized using a latent diffusion model, and then these maps are combined with a physical signal model to generate the final MRI scan. Our experiments demonstrate the efficacy of this approach in generating unseen MR contrasts and preserving physical plausibility. Furthermore, we validate the distributions of generated tissue properties by comparing them to those measured in real brain tissue.
Paper Structure (14 sections, 7 equations, 5 figures, 1 table)

This paper contains 14 sections, 7 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Method
  3. MRI Signal Model
  4. Multimodal Physics-Informed Variational Autoencoder
  5. Latent Diffusion Model
  6. Experiments and Results
  7. Dataset and Preprocessing
  8. Training Details
  9. Results
  10. Synthesis of Unseen Modalities
  11. Analysis of the Tissue Property Distribution
  12. Reconstruction Performance
  13. Limitations
  14. Conclusion

Figures (5)

  • Figure 1: Unobserved tissue property maps generated by our model.
  • Figure 2: Overview of our physics-informed generative model. We combine an MRI signal model with a product-of-experts (PoE) multimodal variational autoencoder and a latent diffusion model.
  • Figure 3: Images of a single brain, represented by the tissue property maps in \ref{['fig:combined']} generated with different signal models and varying acquisition parameters TE-TR-TI. Spin-Echo sequences do not use inversion recovery. The leftmost images use parameters commonly found in the OASIS-3 dataset, whereas the other images use parameter combinations not present in the training data.
  • Figure 4: Distribution of T1 and T2 values in the generated images compared to the median T1 and T2 of white matter (WM) and grey matter (GM) in real brains reported by bojorquez2017normal. For visualization purposes, we clipped the distributions of the generated properties to the 95th percentile. The results show that the introduction of the prior leads to the generation of more realistic property distributions.
  • Figure 5: Images from the data set (top) and their physics-informed VAE reconstructions (bottom), showing the absence of scanner noise in the images generated by our model.