Advances in 3D Generation: A Survey

TL;DR

The paper surveys the rapid expansion of 3D generation, outlining foundational scene representations (explicit, implicit, and hybrids), and organizing generation methods into feedforward, optimization-based, procedural, and novel view synthesis. It highlights key models and strategies, from NeRFs and DMTet to diffusion-based pipelines and 3D-aware GANs, while detailing datasets and applications across humans, faces, and general scenes. By mapping representations to generation paradigms and datasets, the survey identifies open challenges in evaluation, data availability, controllability, and workflow integration, offering a roadmap for future research. The work emphasizes how advances in diffusion, large-scale priors, and hybrid representations are driving increasingly realistic and controllable 3D content, with practical implications for games, film, AR/VR, and metaverse content creation.

Abstract

Generating 3D models lies at the core of computer graphics and has been the focus of decades of research. With the emergence of advanced neural representations and generative models, the field of 3D content generation is developing rapidly, enabling the creation of increasingly high-quality and diverse 3D models. The rapid growth of this field makes it difficult to stay abreast of all recent developments. In this survey, we aim to introduce the fundamental methodologies of 3D generation methods and establish a structured roadmap, encompassing 3D representation, generation methods, datasets, and corresponding applications. Specifically, we introduce the 3D representations that serve as the backbone for 3D generation. Furthermore, we provide a comprehensive overview of the rapidly growing literature on generation methods, categorized by the type of algorithmic paradigms, including feedforward generation, optimization-based generation, procedural generation, and generative novel view synthesis. Lastly, we discuss available datasets, applications, and open challenges. We hope this survey will help readers explore this exciting topic and foster further advancements in the field of 3D content generation.

Figures (12)

• Figure 1: In this survey, we investigate a large variety of 3D generation methods. Over the past decade, 3D generation has achieved remarkable progress and has recently garnered considerable attention due to the success of generative AI in images and videos. 3D generation results from 3D-GAN wu2016learning, DeepSDF park2019deepsdf, DMTet shen2021deep, EG3D chan2022efficient, DreamFusion poole2022dreamfusion, PointE nichol2022point, Zero-1-to-3 liu2023zero and Instant3D li2023instant3d.
• Figure 2: Overview of this survey, including 3D representations, 3D generation methods, datasets and applications. Specifically, we introduce the 3D representations that serve as the backbone for 3D generation. Furthermore, we provide a comprehensive overview of the rapidly growing literature on generation methods, categorized by the type of algorithmic paradigms, including feedforward generation, optimization-based generation, procedural generation, and generative novel view synthesis. Finally, we provide a brief discussion on popular datasets and available applications.
• Figure 3: Neural scene representations used for 3D generation, including explicit, implicit, and hybrid representations. The 3D generation involves the use of scene representations and a differentiable rendering algorithm to create 3D models or render 2D images. On the flip side, these 3D models or 2D images can function as the reconstruction domain or image domain, overseeing the 3D generation of scene representations.
• Figure 4: The evolutionary tree of 3D generation illustrates the primary branch of generation methods and their developments in recent years. Specifically, we provide a comprehensive overview of the rapidly growing literature on generation methods, categorized by the type of algorithmic paradigms, including feedforward generation, optimization-based generation, procedural generation, and generative novel view synthesis.
• Figure 5: Exemplary feedforward 3D generation models. We showcase several representative pipelines of feedforward 3D generation models, including (a) generative adversarial networks, (b) diffusion models, (c) autoregressive models, (d) variational autoencoders and (e) normalizing flows.