Advances in Radiance Field for Dynamic Scene: From Neural Field to Gaussian Field

TL;DR

This survey surveys advances in dynamic scene representation using neural radiance fields and 3D Gaussian splatting, outlining how motion is modeled through rigid, articulated, non-rigid, and hybrid paradigms. It analyzes five primary motion representations (spacetime, canonical deformation, flow, trajectories, and factorization) and discusses how frame-based, canonical-space, and bidirectional approaches trade off temporal consistency and flexibility. The paper highlights auxiliary cues (depth, normals, semantics) and regularization strategies (TV, ARAP, isometric, divergence) that improve plausibility and reconstruction quality, and it discusses the shift toward explicit Gaussian primitives for real-time rendering. It also considers future directions, including scalability to large-scale scenes, integration with generative models and LLMs, and the potential of hybrid representations to achieve high-fidelity, editable 4D digital twins. The work provides a framework and benchmarks to guide researchers entering this rapidly evolving field while pointing to practical implications for autonomous systems, AR/VR, and digital twins.

Abstract

Dynamic scene representation and reconstruction have undergone transformative advances in recent years, catalyzed by breakthroughs in neural radiance fields and 3D Gaussian splatting techniques. While initially developed for static environments, these methodologies have rapidly evolved to address the complexities inherent in 4D dynamic scenes through an expansive body of research. Coupled with innovations in differentiable volumetric rendering, these approaches have significantly enhanced the quality of motion representation and dynamic scene reconstruction, thereby garnering substantial attention from the computer vision and graphics communities. This survey presents a systematic analysis of over 200 papers focused on dynamic scene representation using radiance field, spanning the spectrum from implicit neural representations to explicit Gaussian primitives. We categorize and evaluate these works through multiple critical lenses: motion representation paradigms, reconstruction techniques for varied scene dynamics, auxiliary information integration strategies, and regularization approaches that ensure temporal consistency and physical plausibility. We organize diverse methodological approaches under a unified representational framework, concluding with a critical examination of persistent challenges and promising research directions. By providing this comprehensive overview, we aim to establish a definitive reference for researchers entering this rapidly evolving field while offering experienced practitioners a systematic understanding of both conceptual principles and practical frontiers in dynamic scene reconstruction.

Figures (6)

• Figure 1: Survey at A Glance.(a) Introduction and Foundation. We trace the evolution from static to dynamic scene representation, highlighting the challenges of jointly modeling motion, geometry, and appearance using radiance fields. (b) Motion Representation. We categorize motion patterns and their representation paradigms, examining how they enable complex motion modeling while addressing inherent limitations. (c) Scene Reconstruction. We analyze how motion representations enable scene reconstruction, discussing these methods within a unified framework while investigating how auxiliary information and regularization strategies constrain the learning of radiance fields. (d) Future Trends. We explore promising research directions and how dynamic scene reconstruction could benefit by aid of the rapid development of foundation models and large language models.
• Figure 2: Roadmap of Dynamic Scenes in Radiance Fields. This chronological timeline illustrates the evolution of the field, organizing works into methodological clusters based on their representation paradigms. The representative or first work within each cluster appears in black with accompanying paradigm illustrations, while the dates of remaining works may vary within clusters. Seminal contributions that significantly advanced the field are highlighted with colors.
• Figure 3: A 2D illustration of various motion types.
• Figure 4: Illustration of typical motion representation methods.
• Figure 5: We propose a unified framework to encapsulate various representation paradigms.