Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Comprehensive Survey on 3D Content Generation

Jian Liu, Xiaoshui Huang, Tianyu Huang, Lu Chen, Yuenan Hou, Shixiang Tang, Ziwei Liu, Wanli Ouyang, Wangmeng Zuo, Junjun Jiang, Xianming Liu

TL;DR

3D content generation has seen rapid advances across native 3D methods, 2D-prior-based approaches, and hybrid strategies. The paper proposes a new taxonomy and surveys around 60 papers, highlighting key techniques such as NeRF-based optimization, 3D Gaussian Splatting, and diffusion-driven text-to-3D methods (e.g., DreamFusion). It identifies data, modeling, and evaluation bottlenecks, and outlines open challenges and directions like scalable representations, faster inference, and robust benchmarks. By organizing the field into three interlocked families and providing a resource-rich survey, the work aims to guide both researchers and practitioners in navigating the evolving 3D AIGC landscape.

Abstract

Recent years have witnessed remarkable advances in artificial intelligence generated content(AIGC), with diverse input modalities, e.g., text, image, video, audio and 3D. The 3D is the most close visual modality to real-world 3D environment and carries enormous knowledge. The 3D content generation shows both academic and practical values while also presenting formidable technical challenges. This review aims to consolidate developments within the burgeoning domain of 3D content generation. Specifically, a new taxonomy is proposed that categorizes existing approaches into three types: 3D native generative methods, 2D prior-based 3D generative methods, and hybrid 3D generative methods. The survey covers approximately 60 papers spanning the major techniques. Besides, we discuss limitations of current 3D content generation techniques, and point out open challenges as well as promising directions for future work. Accompanied with this survey, we have established a project website where the resources on 3D content generation research are provided. The project page is available at https://github.com/hitcslj/Awesome-AIGC-3D.

A Comprehensive Survey on 3D Content Generation

TL;DR

3D content generation has seen rapid advances across native 3D methods, 2D-prior-based approaches, and hybrid strategies. The paper proposes a new taxonomy and surveys around 60 papers, highlighting key techniques such as NeRF-based optimization, 3D Gaussian Splatting, and diffusion-driven text-to-3D methods (e.g., DreamFusion). It identifies data, modeling, and evaluation bottlenecks, and outlines open challenges and directions like scalable representations, faster inference, and robust benchmarks. By organizing the field into three interlocked families and providing a resource-rich survey, the work aims to guide both researchers and practitioners in navigating the evolving 3D AIGC landscape.

Abstract

Recent years have witnessed remarkable advances in artificial intelligence generated content(AIGC), with diverse input modalities, e.g., text, image, video, audio and 3D. The 3D is the most close visual modality to real-world 3D environment and carries enormous knowledge. The 3D content generation shows both academic and practical values while also presenting formidable technical challenges. This review aims to consolidate developments within the burgeoning domain of 3D content generation. Specifically, a new taxonomy is proposed that categorizes existing approaches into three types: 3D native generative methods, 2D prior-based 3D generative methods, and hybrid 3D generative methods. The survey covers approximately 60 papers spanning the major techniques. Besides, we discuss limitations of current 3D content generation techniques, and point out open challenges as well as promising directions for future work. Accompanied with this survey, we have established a project website where the resources on 3D content generation research are provided. The project page is available at https://github.com/hitcslj/Awesome-AIGC-3D.
Paper Structure (25 sections, 3 figures)

This paper contains 25 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. 3D Representation
  4. Explicit Representations
  5. Implicit Representations
  6. 2D Diffusion Models
  7. 3D Native Generative Methods
  8. Object
  9. Scene
  10. Human Avatar
  11. 2D Prior-based 3D Generative Methods
  12. Object
  13. Scene
  14. Human Avatar
  15. Hybrid 3D Generative Methods
  16. ...and 10 more sections

Figures (3)

  • Figure 1: Chronological overview of the most relevant 3D native generative methods.
  • Figure 2: Chronological overview of the most relevant 2D prior-based 3D generative methods.
  • Figure 3: Chronological overview of the most relevant hybrid 3D generative methods.