Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generative AI in Vision: A Survey on Models, Metrics and Applications

Gaurav Raut, Apoorv Singh

TL;DR

This survey addresses the problem of comprehensively characterizing generative AI in vision, with a focus on diffusion-based methods and their legacy counterparts. It systematically reviews theoretical foundations (DDPM, score-based diffusion, SDEs) and a broad spectrum of models (GANs, VAEs, autoregressive, normalizing flows, EBMs), situating diffusion models as a versatile framework. The paper highlights key applications—text-to-image, image super-resolution, anomaly detection, and inpainting—and discusses evaluation metrics such as IS, FID, and precision/recall. It also identifies practical challenges (training stability, scalability, interpretability) and outlines future directions, including ethical considerations and physics-inspired approaches, to guide research and deployment in real-world settings.

Abstract

Generative AI models have revolutionized various fields by enabling the creation of realistic and diverse data samples. Among these models, diffusion models have emerged as a powerful approach for generating high-quality images, text, and audio. This survey paper provides a comprehensive overview of generative AI diffusion and legacy models, focusing on their underlying techniques, applications across different domains, and their challenges. We delve into the theoretical foundations of diffusion models, including concepts such as denoising diffusion probabilistic models (DDPM) and score-based generative modeling. Furthermore, we explore the diverse applications of these models in text-to-image, image inpainting, and image super-resolution, along with others, showcasing their potential in creative tasks and data augmentation. By synthesizing existing research and highlighting critical advancements in this field, this survey aims to provide researchers and practitioners with a comprehensive understanding of generative AI diffusion and legacy models and inspire future innovations in this exciting area of artificial intelligence.

Generative AI in Vision: A Survey on Models, Metrics and Applications

TL;DR

This survey addresses the problem of comprehensively characterizing generative AI in vision, with a focus on diffusion-based methods and their legacy counterparts. It systematically reviews theoretical foundations (DDPM, score-based diffusion, SDEs) and a broad spectrum of models (GANs, VAEs, autoregressive, normalizing flows, EBMs), situating diffusion models as a versatile framework. The paper highlights key applications—text-to-image, image super-resolution, anomaly detection, and inpainting—and discusses evaluation metrics such as IS, FID, and precision/recall. It also identifies practical challenges (training stability, scalability, interpretability) and outlines future directions, including ethical considerations and physics-inspired approaches, to guide research and deployment in real-world settings.

Abstract

Generative AI models have revolutionized various fields by enabling the creation of realistic and diverse data samples. Among these models, diffusion models have emerged as a powerful approach for generating high-quality images, text, and audio. This survey paper provides a comprehensive overview of generative AI diffusion and legacy models, focusing on their underlying techniques, applications across different domains, and their challenges. We delve into the theoretical foundations of diffusion models, including concepts such as denoising diffusion probabilistic models (DDPM) and score-based generative modeling. Furthermore, we explore the diverse applications of these models in text-to-image, image inpainting, and image super-resolution, along with others, showcasing their potential in creative tasks and data augmentation. By synthesizing existing research and highlighting critical advancements in this field, this survey aims to provide researchers and practitioners with a comprehensive understanding of generative AI diffusion and legacy models and inspire future innovations in this exciting area of artificial intelligence.
Paper Structure (25 sections, 22 equations, 3 figures, 1 table)

This paper contains 25 sections, 22 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Generative Models in Vision
  3. Diffusion models
  4. Denoising Diffusion Probabilistic Models
  5. Noise Conditional Score Models
  6. Stochastic Differential Equations Generative Models
  7. Generative Adversarial Networks
  8. Variational Autoencoders
  9. Autoregressive models
  10. Normalizing flow models
  11. Energy Based models
  12. Evaluation metrics for vision generative models
  13. Inception Score (IS)
  14. Fréchet Inception Distance
  15. Precision and Recall
  16. ...and 10 more sections

Figures (3)

  • Figure 1: a) Images generated using stable diffusionrombach2022highresolution; b) Image super-resolution results from SR3saharia2021image; c) Image inpainting results from Palettesaharia2022palette
  • Figure 2: An extension of generative models classification based ongoodfellow2017nips
  • Figure 3: An example of a conditional generative model: Latent diffusion modelrombach2022highresolution