DynoSurf: Neural Deformation-based Temporally Consistent Dynamic Surface Reconstruction

TL;DR

DynoSurf tackles temporally-consistent dynamic surface reconstruction from dynamic point clouds without correspondences or ground-truth supervision. It combines a coarse-to-fine, deformable tetrahedron template surface with a learnable control-point deformation field, enabling per-frame and cross-frame alignment through a differentiable pipeline that includes DMT for mesh extraction and a blended deformation mechanism. The method introduces a robust loss set and an adaptive keyframe-based template learning strategy, achieving state-of-the-art results on DFAUST, DT4D, and AMA without supervision and showing resilience to noise and missing data. This approach offers a practical, unsupervised tool for dynamic mesh reconstruction with broad applicability in motion capture, VR/AR, and animation workflows.

Abstract

This paper explores the problem of reconstructing temporally consistent surfaces from a 3D point cloud sequence without correspondence. To address this challenging task, we propose DynoSurf, an unsupervised learning framework integrating a template surface representation with a learnable deformation field. Specifically, we design a coarse-to-fine strategy for learning the template surface based on the deformable tetrahedron representation. Furthermore, we propose a learnable deformation representation based on the learnable control points and blending weights, which can deform the template surface non-rigidly while maintaining the consistency of the local shape. Experimental results demonstrate the significant superiority of DynoSurf over current state-of-the-art approaches, showcasing its potential as a powerful tool for dynamic mesh reconstruction. The code is publicly available at https://github.com/yaoyx689/DynoSurf.

Figures (17)

• Figure 1: Temporally-consistent dynamic meshes (i.e., vertices are corresponded and connections are identical over time) reconstructed by our DynoSurf from continuous dynamic 3D point cloud sequences without using any shape-prior, ground-truth surface, and ground-truth temporal correspondence. The color and texture map are used to illustrate correspondence across reconstructed mesh frames.
• Figure 2: Illustration of the proposed DynoSurf, which can reconstruct from continuous dynamic point cloud sequences temporally-consistent dynamic surfaces without requiring any ground-truth surface and temporal correspondence information.
• Figure 3: Illustration of the pipeline of learning template surface via deformable tetrahedron.
• Figure 4: Illustration of the proposed control points blending-based learnable deformation stage for temporal reconstruction. Note that the adaptively enhanced template surface will be deformed to all frames.
• Figure 5: Comparison of visual results by different methods on the DFAUST dataset.