Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Physics-Inspired Generative Models in Medical Imaging: A Review

Dennis Hein, Afshin Bozorgpour, Dorit Merhof, Ge Wang

TL;DR

This review surveys physics-inspired generative models for medical imaging, focusing on diffusion models (DDPMs, SDMs) and Poisson Flow Generative Models (PFGMs, PFGM++), and their applications across reconstruction, generation, and analysis. It details the theoretical underpinnings of DDPMs, NCSNs, SDMs, PFMs, and the unified PFGM++ framework, with acceleration strategies such as DDIMs and consistency models. It covers practical medical-imaging tasks including CT/MRI/PET reconstruction, high-quality image synthesis, segmentation-guided generation, and robustness to data imbalance, supported by several novel frameworks and architectures. The future directions emphasize unification of SDMs and PFMs, integration with vision-language models and foundation models, and addressing data privacy and computational costs, while highlighting challenges like biases and hallucinations. Overall, physics-inspired GMs provide principled priors that enable unsupervised reconstruction, high-fidelity synthesis, and flexible data generation with potential to transform clinical workflows.

Abstract

Physics-inspired Generative Models (GMs), in particular Diffusion Models (DMs) and Poisson Flow Models (PFMs), enhance Bayesian methods and promise great utility in medical imaging. This review examines the transformative role of such generative methods. First, a variety of physics-inspired GMs, including Denoising Diffusion Probabilistic Models (DDPMs), Score-based Diffusion Models (SDMs), and Poisson Flow Generative Models (PFGMs and PFGM++), are revisited, with an emphasis on their accuracy, robustness as well as acceleration. Then, major applications of physics-inspired GMs in medical imaging are presented, comprising image reconstruction, image generation, and image analysis. Finally, future research directions are brainstormed, including unification of physics-inspired GMs, integration with Vision-Language Models (VLMs), and potential novel applications of GMs. Since the development of generative methods has been rapid, this review will hopefully give peers and learners a timely snapshot of this new family of physics-driven generative models and help capitalize their enormous potential for medical imaging.

Physics-Inspired Generative Models in Medical Imaging: A Review

TL;DR

This review surveys physics-inspired generative models for medical imaging, focusing on diffusion models (DDPMs, SDMs) and Poisson Flow Generative Models (PFGMs, PFGM++), and their applications across reconstruction, generation, and analysis. It details the theoretical underpinnings of DDPMs, NCSNs, SDMs, PFMs, and the unified PFGM++ framework, with acceleration strategies such as DDIMs and consistency models. It covers practical medical-imaging tasks including CT/MRI/PET reconstruction, high-quality image synthesis, segmentation-guided generation, and robustness to data imbalance, supported by several novel frameworks and architectures. The future directions emphasize unification of SDMs and PFMs, integration with vision-language models and foundation models, and addressing data privacy and computational costs, while highlighting challenges like biases and hallucinations. Overall, physics-inspired GMs provide principled priors that enable unsupervised reconstruction, high-fidelity synthesis, and flexible data generation with potential to transform clinical workflows.

Abstract

Physics-inspired Generative Models (GMs), in particular Diffusion Models (DMs) and Poisson Flow Models (PFMs), enhance Bayesian methods and promise great utility in medical imaging. This review examines the transformative role of such generative methods. First, a variety of physics-inspired GMs, including Denoising Diffusion Probabilistic Models (DDPMs), Score-based Diffusion Models (SDMs), and Poisson Flow Generative Models (PFGMs and PFGM++), are revisited, with an emphasis on their accuracy, robustness as well as acceleration. Then, major applications of physics-inspired GMs in medical imaging are presented, comprising image reconstruction, image generation, and image analysis. Finally, future research directions are brainstormed, including unification of physics-inspired GMs, integration with Vision-Language Models (VLMs), and potential novel applications of GMs. Since the development of generative methods has been rapid, this review will hopefully give peers and learners a timely snapshot of this new family of physics-driven generative models and help capitalize their enormous potential for medical imaging.
Paper Structure (27 sections, 31 equations, 3 figures)

This paper contains 27 sections, 31 equations, 3 figures.

Table of Contents

  1. INTRODUCTION
  2. PHYSICS-INSPIRED GENERATIVE MODELS
  3. DDPMs
  4. NCSNs
  5. SDMs
  6. PFGMs
  7. PFGM++
  8. PFGM++ and SDMs in theory and in practice
  9. Trade-off robustness for rigidity by varying $D$
  10. Latent DMs (LDMs)
  11. Accelerated Sampling Schemes
  12. APPLICATIONS IN MEDICAL IMAGING
  13. Image Reconstruction
  14. CT Reconstruction
  15. MRI Reconstruction
  16. ...and 12 more sections

Figures (3)

  • Figure 1: Scopus$^{\text{TM}}$ search results on diffusion-type generative AI models. (a) The number of diffusion-type generative AI papers; (b) the number of citations of these papers; (c) applications of diffusion-type generative AI models in various fields; (d) contributions from top countries and territories; and (e) a VOSviewer$^{\text{TM}}$ presentation in terms of authors' key-words co-occurrences. With a threshold of 5 co-occurrences, 258 key words were selected to form this graph network.
  • Figure 2: Physics-inspired deep GMs. (a) Schematic of forward and reverse SDE on SDMs song2021; (b) Overview of PFGM++ xu2023; (c) Schematic overview of LDM rombach2022; (d) Schematic of mapping learned by consistency models song2023. Note that different notation may have been used in the text compared to this figure.
  • Figure 3: Visualization of three exemplary measurement processes for CT and MRI. (a) limited-angle CT (LA-CT); (b) sparse-view CT (SV-CT); and (c) compressed sensing-based MRI (CS-MRI). Adapted from chung2023solving.