Neu-PiG: Neural Preconditioned Grids for Fast Dynamic Surface Reconstruction on Long Sequences

TL;DR

Neu-PiG is presented, a fast deformation optimization method based on a novel preconditioned latent-grid encoding that distributes spatial features parameterized on the position and normal direction of a keyframe surface to achieve high-fidelity, drift-free surface reconstructions in seconds.

Abstract

Temporally consistent surface reconstruction of dynamic 3D objects from unstructured point cloud data remains challenging, especially for very long sequences. Existing methods either optimize deformations incrementally, risking drift and requiring long runtimes, or rely on complex learned models that demand category-specific training. We present Neu-PiG, a fast deformation optimization method based on a novel preconditioned latent-grid encoding that distributes spatial features parameterized on the position and normal direction of a keyframe surface. Our method encodes entire deformations across all time steps at various spatial scales into a multi-resolution latent grid, parameterized by the position and normal direction of a reference surface from a single keyframe. This latent representation is then augmented for time modulation and decoded into per-frame 6-DoF deformations via a lightweight multilayer perceptron (MLP). To achieve high-fidelity, drift-free surface reconstructions in seconds, we employ Sobolev preconditioning during gradient-based training of the latent space, completely avoiding the need for any explicit correspondences or further priors. Experiments across diverse human and animal datasets demonstrate that Neu-PiG outperforms state-the-art approaches, offering both superior accuracy and scalability to long sequences while running at least 60x faster than existing training-free methods and achieving inference speeds on the same order as heavy pretrained models.

Figures (9)

• Figure 1: We present Neu-PiG, a method that learns spatially smooth and temporally coherent deformations from dynamic point clouds. Starting from input point clouds (left), a unified latent space parameterized by an initial reference mesh (middle) is optimized through multi-scale grids to produce high-fidelity deformations (right) without relying on strong priors or correspondences.
• Figure 2: Overview of Neu-PiG. A reference surface $\mathcal{X}_{t_{\mathrm{key}}}$ is first generated from the input point cloud at a keyframe $\mathcal{P}_{t_{\mathrm{key}}}$. Position and normal-direction-embedded latent features $\bm{z}_{\mathrm{p}}$ and $\bm{z}_{\mathrm{n}}$ are then sampled from multi-resolution preconditioned grids and combined with a time embedding $\bm{\gamma}(t)$. A lightweight MLP $\psi$ predicts per-frame 6-DoF deformations that are applied to $\mathcal{X}_{t_{\mathrm{key}}}$ to reconstruct each frame $\hat{\mathcal{X}}_t$. Sobolev preconditioning in the latent space enforces spatial smoothness and temporal coherence across the sequence during optimization.
• Figure 3: Qualitative reconstructions on DT4D. Neu-PiG preserves geometry and temporal consistency across complex animal motions.
• Figure 4: Convergence of Chamfer Distance and Normal Consistency on DT4D, showing high-quality reconstructions in seconds.
• Figure 5: Performance across sequence lengths on AMA. Neu-PiG maintains accuracy and stability as duration increases.