Advances in 4D Generation: A Survey

TL;DR

The survey investigates the rapid emergence of 4D generation, which synthesizes temporally coherent dynamic 3D content guided by user input. It surveys fundamental 4D representations (mesh, NeRF, point cloud, Gaussian splatting), foundational techniques (diffusion models and Score Distillation Sampling), and four generative paradigms (End-to-End, Generated-Data-Based, Implicit-Distillation-Based, Explicit-Supervision-Based). It then discusses conditioning modalities, applications across objects, scenes, digital humans, editing, and autonomous driving, and identifies five core challenges: consistency, controllability, diversity, efficiency, and fidelity. Finally, the paper outlines future directions, including large multimodal 4D datasets, unified efficient frameworks, and standardized benchmarks to accelerate progress and practical deployment of 4D generation technologies.

Abstract

Generative artificial intelligence has recently progressed from static image and video synthesis to 3D content generation, culminating in the emergence of 4D generation-the task of synthesizing temporally coherent dynamic 3D assets guided by user input. As a burgeoning research frontier, 4D generation enables richer interactive and immersive experiences, with applications ranging from digital humans to autonomous driving. Despite rapid progress, the field lacks a unified understanding of 4D representations, generative frameworks, basic paradigms, and the core technical challenges it faces. This survey provides a systematic and in-depth review of the 4D generation landscape. To comprehensively characterize 4D generation, we first categorize fundamental 4D representations and outline associated techniques for 4D generation. We then present an in-depth analysis of representative generative pipelines based on conditions and representation methods. Subsequently, we discuss how motion and geometry priors are integrated into 4D outputs to ensure spatio-temporal consistency under various control schemes. From an application perspective, this paper summarizes 4D generation tasks in areas such as dynamic object/scene generation, digital human synthesis, editable 4D content, and embodied AI. Furthermore, we summarize and multi-dimensionally compare four basic paradigms for 4D generation: End-to-End, Generated-Data-Based, Implicit-Distillation-Based, and Explicit-Supervision-Based. Concluding our analysis, we highlight five key challenges-consistency, controllability, diversity, efficiency, and fidelity-and contextualize these with current approaches.By distilling recent advances and outlining open problems, this work offers a comprehensive and forward-looking perspective to guide future research in 4D generation.

Figures (7)

• Figure 1: 4D generation development trends. The field of 4D generation has grown rapidly, as shown by the sharp increase in annual publications. Representative works from prominent academic and industrial institutions—such as NVIDIA, Snap Inc., TUM, and VITA Lab—highlight the expanding diversity and increasing impact of this emerging research area.
• Figure 2: Overall of the survey. A systematic taxonomy illustrating key aspects of 4D generation, including representation methods, foundational techniques, conditioning strategies, diverse applications, representative paradigms, and core technical challenges. This structured framework clearly delineates current relationships and emerging research directions within the field.
• Figure 3: Illustration of four major 4D representation methods: (A) 4D Mesh explicitly represents evolving surfaces by continuously updating vertex positions and connectivity; (B) 4D NeRF implicitly encodes scenes as continuous volumetric functions, providing smooth renderings across views and time; (C) 4D Point Cloud represents dynamics via discrete point displacements, offering flexibility and simplicity; (D) 4D Gaussian Splatting employs anisotropic Gaussian primitives for compact, high-quality representation and rendering. While 4D NeRF employs an implicit representation approach, the other three methods explicitly encode geometric entities.
• Figure 4: 4D generation vs. related generative paradigms. Image generation methodologies focus on synthesizing static, single-viewpoint imagery. Video generation incorporates temporal dynamics, typically focusing on content viewed primarily from a single or constrained set of viewpoints. 3D generation synthesizes static geometric models, while multiview generation simultaneously produces images across multiple viewpoints at a single time instance. Distinctively, 4D generation synthesizes dynamic assets that integrate both spatio-temporal coherence and multiview observability.
• Figure 5: Representative directions in 4D generation. Based on different control modalities, 4D generation tasks are categorized into five key domains: (A) Text-to-4D Generation, where methods such as 4D-fy bahmani20244dfy, MAV3D singer2023text, and AYG ling2024align enable the generation of diverse 4D assets using text as the control condition; (B) Image-to-4D Generation, exemplified by DreamGaussian4D ren2023dreamgaussian4d (DG4D) and Human4DiT shao_human4dit_2024, which focuses on faithfully reconstructing 4D assets from input images; (C) Video-to-4D Generation, as demonstrated by 4Diffusion zhang_4diffusion_2024 and L4GM ren2024l4gm, emphasizes maintaining spatial consistency over time in generated 4D sequences; (D) 3D-to-4D Generation, like HyperDiffusion dou_dynamic_2024, extends static 3D assets into the temporal dimension to create dynamic 4D outputs; (E) Multi-conditional 4D Generation, showcased by TC4D bahmani_tc4d_2024, STAR4D chai_star_2024 and Sync4D fu_sync4d_2024, integrates multiple control conditions to achieve precise and controllable 4D generation.