Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Text-to-Image Synthesis: A Decade Survey

Nonghai Zhang, Hao Tang

TL;DR

This survey discusses the development of GANs, autoregressive models, and diffusion models for T2I, focusing on their generative capabilities and diversity when conditioned on text, and explores cutting-edge research on various aspects of T2I.

Abstract

When humans read a specific text, they often visualize the corresponding images, and we hope that computers can do the same. Text-to-image synthesis (T2I), which focuses on generating high-quality images from textual descriptions, has become a significant aspect of Artificial Intelligence Generated Content (AIGC) and a transformative direction in artificial intelligence research. Foundation models play a crucial role in T2I. In this survey, we review over 440 recent works on T2I. We start by briefly introducing how GANs, autoregressive models, and diffusion models have been used for image generation. Building on this foundation, we discuss the development of these models for T2I, focusing on their generative capabilities and diversity when conditioned on text. We also explore cutting-edge research on various aspects of T2I, including performance, controllability, personalized generation, safety concerns, and consistency in content and spatial relationships. Furthermore, we summarize the datasets and evaluation metrics commonly used in T2I research. Finally, we discuss the potential applications of T2I within AIGC, along with the challenges and future research opportunities in this field.

Text-to-Image Synthesis: A Decade Survey

TL;DR

This survey discusses the development of GANs, autoregressive models, and diffusion models for T2I, focusing on their generative capabilities and diversity when conditioned on text, and explores cutting-edge research on various aspects of T2I.

Abstract

When humans read a specific text, they often visualize the corresponding images, and we hope that computers can do the same. Text-to-image synthesis (T2I), which focuses on generating high-quality images from textual descriptions, has become a significant aspect of Artificial Intelligence Generated Content (AIGC) and a transformative direction in artificial intelligence research. Foundation models play a crucial role in T2I. In this survey, we review over 440 recent works on T2I. We start by briefly introducing how GANs, autoregressive models, and diffusion models have been used for image generation. Building on this foundation, we discuss the development of these models for T2I, focusing on their generative capabilities and diversity when conditioned on text. We also explore cutting-edge research on various aspects of T2I, including performance, controllability, personalized generation, safety concerns, and consistency in content and spatial relationships. Furthermore, we summarize the datasets and evaluation metrics commonly used in T2I research. Finally, we discuss the potential applications of T2I within AIGC, along with the challenges and future research opportunities in this field.

Paper Structure

This paper contains 43 sections, 7 equations, 28 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Basic models of T2I
  3. Generative Adversarial Networks (GAN)
  4. Autoregressive Models (AR)
  5. Diffusion Models (DM)
  6. Development of GAN, AR, and DM in T2I
  7. Developments of GAN Models in T2I
  8. Developments of AR Models in T2I
  9. Rapid Rise of Diffusion Models in T2I
  10. Recent Research Directions in T2I
  11. Alignment and Human Feedback
  12. Personalized Generation
  13. Controllable T2I Generation
  14. T2I Style Transfer
  15. Text-guided Image Generation
  16. ...and 28 more sections

Figures (28)

  • Figure 1: Representative works on the text-to-image synthesis task over time are shown. The GAN-based methods, autoregressive methods, and diffusion-based methods are highlighted in orange, blue, and red, respectively.
  • Figure 2: The text-conditional convolutional GAN architecturereed2016generative. Text encoding $p(t)$ is used by both generator and discriminator. It is projected to lower dimensions and depth concatenated with image feature maps for further stages of convolutional processing.
  • Figure 3: The training of DALL-Eramesh2021zero is divided into two stages. The first stage trains the codebook of VQ-VAE, while the second stage trains the Transformer, corresponding to Stage One and Stage Two indicated in the figure.
  • Figure 4: The framework of CogViewding2021cogview. [ROI1], [BASE1], etc., are seperator tokens.
  • Figure 5: DDPMho2020denoising generates new images by adding noise and then denoising.
  • ...and 23 more figures