Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Text to Image Generation and Editing: A Survey

Pengfei Yang, Ngai-Man Cheung, Xinda Ma

TL;DR

This survey consolidates 141 text-to-image generation and editing works from 2021–2024 across autoregressive, non-autoregressive, GAN, and diffusion paradigms, detailing architectures, prompts/embeddings, training strategies, datasets, and evaluation metrics. It contrasts generation and editing approaches, discusses social impacts, and proposes future directions including model scaling, prompt/embedding optimization, and hybrid architectures. The work highlights diffusion-dominant trends, latent-space strategies (e.g., LDM), and cross-attention-based conditioning as central to performance and controllability. It also emphasizes practical concerns such as data quality, computation, and safety, offering insights for researchers to push toward higher fidelity, fidelity-to-prompt, and safer deployment in real-world settings.

Abstract

Text-to-image generation (T2I) refers to the text-guided generation of high-quality images. In the past few years, T2I has attracted widespread attention and numerous works have emerged. In this survey, we comprehensively review 141 works conducted from 2021 to 2024. First, we introduce four foundation model architectures of T2I (autoregression, non-autoregression, GAN and diffusion) and the commonly used key technologies (autoencoder, attention and classifier-free guidance). Secondly, we systematically compare the methods of these studies in two directions, T2I generation and T2I editing, including the encoders and the key technologies they use. In addition, we also compare the performance of these researches side by side in terms of datasets, evaluation metrics, training resources, and inference speed. In addition to the four foundation models, we survey other works on T2I, such as energy-based models and recent Mamba and multimodality. We also investigate the potential social impact of T2I and provide some solutions. Finally, we propose unique insights of improving the performance of T2I models and possible future development directions. In summary, this survey is the first systematic and comprehensive overview of T2I, aiming to provide a valuable guide for future researchers and stimulate continued progress in this field.

Text to Image Generation and Editing: A Survey

TL;DR

This survey consolidates 141 text-to-image generation and editing works from 2021–2024 across autoregressive, non-autoregressive, GAN, and diffusion paradigms, detailing architectures, prompts/embeddings, training strategies, datasets, and evaluation metrics. It contrasts generation and editing approaches, discusses social impacts, and proposes future directions including model scaling, prompt/embedding optimization, and hybrid architectures. The work highlights diffusion-dominant trends, latent-space strategies (e.g., LDM), and cross-attention-based conditioning as central to performance and controllability. It also emphasizes practical concerns such as data quality, computation, and safety, offering insights for researchers to push toward higher fidelity, fidelity-to-prompt, and safer deployment in real-world settings.

Abstract

Text-to-image generation (T2I) refers to the text-guided generation of high-quality images. In the past few years, T2I has attracted widespread attention and numerous works have emerged. In this survey, we comprehensively review 141 works conducted from 2021 to 2024. First, we introduce four foundation model architectures of T2I (autoregression, non-autoregression, GAN and diffusion) and the commonly used key technologies (autoencoder, attention and classifier-free guidance). Secondly, we systematically compare the methods of these studies in two directions, T2I generation and T2I editing, including the encoders and the key technologies they use. In addition, we also compare the performance of these researches side by side in terms of datasets, evaluation metrics, training resources, and inference speed. In addition to the four foundation models, we survey other works on T2I, such as energy-based models and recent Mamba and multimodality. We also investigate the potential social impact of T2I and provide some solutions. Finally, we propose unique insights of improving the performance of T2I models and possible future development directions. In summary, this survey is the first systematic and comprehensive overview of T2I, aiming to provide a valuable guide for future researchers and stimulate continued progress in this field.
Paper Structure (56 sections, 22 equations, 3 figures, 1 table)

This paper contains 56 sections, 22 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Foundation Models
  4. Autoregression
  5. Non-Autoregression (NAR)
  6. GAN
  7. Diffusion
  8. Key Strategies
  9. Autoencoder
  10. Attention
  11. Class-free Guidance
  12. Image Generation
  13. Autoregression
  14. Architecture
  15. Training Strategies
  16. ...and 41 more sections

Figures (3)

  • Figure 1: Architecture of this survey.
  • Figure 2: Taxonomy of image generation. Papers are categorized according to its main contribution. Note that papers falling into other categories according to its sub-important contributions are shown in bold and the description of which are not given again for concise.
  • Figure 3: Taxonomy of image editing. Papers are categorized according to its main contribution. Note that papers falling into other categories according to its sub-important contributions are shown in bold and the description of which are not given again for concise.