Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

3D Brain and Heart Volume Generative Models: A Survey

Yanbin Liu, Girish Dwivedi, Farid Boussaid, Mohammed Bennamoun

TL;DR

This survey addresses the lack of comprehensive 3D volume analyses in medical generative modeling by focusing on brain and heart structures. It introduces a taxonomy that separates unconditional and conditional generative models and reviews seven applications: unconditional synthesis, classification, conditional synthesis, segmentation, denoising, detection, and registration, with detailed organ- and task-specific tables. The discussion covers GANs, VAEs, autoregressive models, and diffusion-based approaches, highlighting data scarcity, computational demands, and the promise of diffusion models and self-supervised learning as future directions. The work provides practical guidance and resources (GitHub) to track evolving methods, supporting researchers and developers in 3D medical volume generation and its clinical translation.

Abstract

Generative models such as generative adversarial networks and autoencoders have gained a great deal of attention in the medical field due to their excellent data generation capability. This paper provides a comprehensive survey of generative models for three-dimensional (3D) volumes, focusing on the brain and heart. A new and elaborate taxonomy of unconditional and conditional generative models is proposed to cover diverse medical tasks for the brain and heart: unconditional synthesis, classification, conditional synthesis, segmentation, denoising, detection, and registration. We provide relevant background, examine each task and also suggest potential future directions. A list of the latest publications will be updated on Github to keep up with the rapid influx of papers at https://github.com/csyanbin/3D-Medical-Generative-Survey.

3D Brain and Heart Volume Generative Models: A Survey

TL;DR

This survey addresses the lack of comprehensive 3D volume analyses in medical generative modeling by focusing on brain and heart structures. It introduces a taxonomy that separates unconditional and conditional generative models and reviews seven applications: unconditional synthesis, classification, conditional synthesis, segmentation, denoising, detection, and registration, with detailed organ- and task-specific tables. The discussion covers GANs, VAEs, autoregressive models, and diffusion-based approaches, highlighting data scarcity, computational demands, and the promise of diffusion models and self-supervised learning as future directions. The work provides practical guidance and resources (GitHub) to track evolving methods, supporting researchers and developers in 3D medical volume generation and its clinical translation.

Abstract

Generative models such as generative adversarial networks and autoencoders have gained a great deal of attention in the medical field due to their excellent data generation capability. This paper provides a comprehensive survey of generative models for three-dimensional (3D) volumes, focusing on the brain and heart. A new and elaborate taxonomy of unconditional and conditional generative models is proposed to cover diverse medical tasks for the brain and heart: unconditional synthesis, classification, conditional synthesis, segmentation, denoising, detection, and registration. We provide relevant background, examine each task and also suggest potential future directions. A list of the latest publications will be updated on Github to keep up with the rapid influx of papers at https://github.com/csyanbin/3D-Medical-Generative-Survey.
Paper Structure (55 sections, 10 equations, 7 figures, 8 tables)

This paper contains 55 sections, 10 equations, 7 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Related Survey
  4. Unconditional and Conditional Generative Models
  5. Generative Adversarial Networks
  6. Vanilla GAN
  7. Conditional GAN
  8. Pixel-to-pixel
  9. CycleGAN
  10. WGAN and WGAN-GP
  11. StyleGAN and StyleGAN2
  12. Autoencoders
  13. Variational Autoencoder (VAE)
  14. Conditional VAE (CVAE)
  15. U-Net
  16. ...and 40 more sections

Figures (7)

  • Figure 1: Statistics of the 3D brain and heart volume generative models. (a) Statistics of all publications according to medical applications. (b) Categorization by year of publication (2017-2022). Uncond. Syn.: Unconditional Synthesis, Cond. Syn.: Conditional Synthesis.
  • Figure 2: Proposed taxonomy of 3D generative models for the brain and the heart. The numbers in the parentheses denote the publication year (after 2000).
  • Figure 3: Generation process in unconditional and conditional generative models.
  • Figure 4: The structure of (a) the vanilla (unconditional) and (b) the conditional GAN.
  • Figure 5: The structure of Variational Autoencoder (VAE).
  • ...and 2 more figures