DynaSurfGS: Dynamic Surface Reconstruction with Planar-based Gaussian Splatting

TL;DR

DynaSurfGS tackles the dual challenge of high-fidelity dynamic surface reconstruction and photorealistic rendering from sparse multi-view data. It introduces a Hex-Plane-based deformation field together with planar-based Gaussian splatting, augmented by normal regularization and ARAP constraints to ensure smooth geometry and temporal rigidity. The method optimizes a combined loss including photometric, normal, and ARAP terms, with a gradual regularization schedule, achieving state-of-the-art results on D-NeRF, DG-Mesh, and Ub4D datasets. This work advances practical dynamic scene reconstruction by linking accurate geometry with realistic rendering, enabling robust dynamic surface capture from monocular or sparse data.

Abstract

Dynamic scene reconstruction has garnered significant attention in recent years due to its capabilities in high-quality and real-time rendering. Among various methodologies, constructing a 4D spatial-temporal representation, such as 4D-GS, has gained popularity for its high-quality rendered images. However, these methods often produce suboptimal surfaces, as the discrete 3D Gaussian point clouds fail to align with the object's surface precisely. To address this problem, we propose DynaSurfGS to achieve both photorealistic rendering and high-fidelity surface reconstruction of dynamic scenarios. Specifically, the DynaSurfGS framework first incorporates Gaussian features from 4D neural voxels with the planar-based Gaussian Splatting to facilitate precise surface reconstruction. It leverages normal regularization to enforce the smoothness of the surface of dynamic objects. It also incorporates the as-rigid-as-possible (ARAP) constraint to maintain the approximate rigidity of local neighborhoods of 3D Gaussians between timesteps and ensure that adjacent 3D Gaussians remain closely aligned throughout. Extensive experiments demonstrate that DynaSurfGS surpasses state-of-the-art methods in both high-fidelity surface reconstruction and photorealistic rendering.

Figures (11)

• Figure 1: We propose the DynaSurfGS framework, which can facilitate real-time photorealistic rendering and dynamic high-fidelity surface reconstruction. Compared with recent SOTA methods, such as DG-mesh liu2024dynamic, MaGS ma2024reconstructing, and 4D-GS wu20244d, DynaSurfGS achieves smooth surfaces with meticulous geometry.
• Figure 2: Overview of our method. Firstly, in the deformation field, we represent the spatial and temporal information of dynamic objects in Hex-Plane and use an MLP to estimate the 3D Gaussian deformation. Subsequently, ARAP regularization is applied to ensure the local rigidity of the dynamic object at different moments. Finally, planar-based Gaussian splatting is used to obtain the unbiased depth map and render the transformed 3D Gaussian to images.
• Figure 3: Qualitative comparison on the DG-Mesh dataset. We present the visualizations of the reconstructed meshes and the rendered images. Our method demonstrates superior geometry and appearance compared to other baselines, which often display incomplete or noisy surfaces.
• Figure 4: Qualitative comparison on the D-NeRF dataset. We present the visualizations of the reconstructed meshes and the rendering images. Our method demonstrates superior geometry and appearance compared to other baselines, which often display incomplete or noisy surfaces.
• Figure 5: Qualitative comparison on the Ub4D dataset. We provide the visualizations of the reconstructed meshes on the real data. Compared to other methods, our method is better at reconstructing geometric details such as eyes.