Dynamic Neural Surfaces for Elastic 4D Shape Representation and Analysis

TL;DR

This work addresses the challenge of statistically analyzing genus-0 4D surfaces (3D shapes evolving in time) by treating them as continuous functions rather than discretized meshes. It introduces Dynamic Spherical Neural Surfaces (D-SNS), a neural implicit representation $F_\Theta: \mathbb{S}^2 \times [0,1] \to [-1,1]^3$ learned by overfitting to discrete 4D data, enabling direct computation of differential quantities via automatic differentiation. By mapping surfaces into the Square Root Normal Field (SRNF) and Square Root Velocity Field (SRVF) spaces, the authors obtain $\mathbb{L}^{2}$-friendly metrics that simplify spatiotemporal registration, geodesics, and mean computation, while preserving the continuous nature of the 4D surfaces. The framework is demonstrated on 4D human bodies and faces, achieving high-fidelity representations, robust registration, and plausible 4D means, with practical implications for functional 4D shape analysis. Limitations include restriction to genus-0 closed surfaces and substantial computation time per surface, suggesting future work toward genus generalization and more scalable, shape-agnostic representations.

Abstract

We propose a novel framework for the statistical analysis of genus-zero 4D surfaces, i.e., 3D surfaces that deform and evolve over time. This problem is particularly challenging due to the arbitrary parameterizations of these surfaces and their varying deformation speeds, necessitating effective spatiotemporal registration. Traditionally, 4D surfaces are discretized, in space and time, before computing their spatiotemporal registrations, geodesics, and statistics. However, this approach may result in suboptimal solutions and, as we demonstrate in this paper, is not necessary. In contrast, we treat 4D surfaces as continuous functions in both space and time. We introduce Dynamic Spherical Neural Surfaces (D-SNS), an efficient smooth and continuous spatiotemporal representation for genus-0 4D surfaces. We then demonstrate how to perform core 4D shape analysis tasks such as spatiotemporal registration, geodesics computation, and mean 4D shape estimation, directly on these continuous representations without upfront discretization and meshing. By integrating neural representations with classical Riemannian geometry and statistical shape analysis techniques, we provide the building blocks for enabling full functional shape analysis. We demonstrate the efficiency of the framework on 4D human and face datasets. The source code and additional results are available at https://4d-dsns.github.io/DSNS/.

Figures (23)

• Figure 1: Dynamic Spherical Neural Surface. Given a discrete 4D surface of meshes parameterized by a unit sphere $\mathbb{S}^2$ and time $t$ - we overfit an MLP $F_\Theta$ to create a continuous 4D surface $F$ by minimizing the Mean Square Error between the ground truth and predicted surface points.
• Figure 2: Illustration of the spatiotemporal registration framework proposed in this paper. "Enc" refers to positional encoding.
• Figure 3: Illustration of the spatial registration of Dynamic Spherical Neural Surfaces (D-SNSs). First, input D-SNSs are mapped to their SRNF representation. We then use the $\mathbb{L}^{2}$ metric in the SRNF space to elastically register the two surfaces, i.e., finding the optimal rotation $O^*$ and diffeomorphism $\gamma^*$ that minimize the $\mathbb{L}^{2}$ metric in the SRNF space.
• Figure 4: A spatial diffeomorphism can be formulated as an MLP with two layers: the input layer evaluates the harmonic basis at the query point $s\in \mathbb{S}^2$. The output layer then computes their weighted sum. The learnable parameters are the weights of the output layer. To perform spatial registration, we freeze the weights of the D-SNS networks and only optimize the weights of the diffeomorphism network using the loss of Eqn. \ref{['eq:spatialregistrationsrnfspace']}.
• Figure 5: Illustration of the temporal registration process of two 4D surfaces $F_1$ and $F_2$ using the proposed Dynamic Spherical Neural Surfaces representation. The 4D surfaces are first mapped to the SRVF space, which has an Eucldiean structure. Thus, we formulate the temporal registration of 4D surfaces as that of elastic registration of curves in the SRVF space.