NeuSpring: Neural Spring Fields for Reconstruction and Simulation of Deformable Objects from Videos

TL;DR

NeuSpring addresses deformable-object reconstruction and future prediction from videos by marrying physics-based spring-mass modeling with a neural representation. It introduces a piecewise topology approach to capture material heterogeneity and a canonical-coordinate neural spring field with a tri-plane architecture to learn spring properties across time. A two-stage optimization jointly learns topology and physics, then aligns rendered images with observations, yielding superior reconstruction and predictive capabilities on the PhysTwin dataset, with $CD$ improvements of $20\%$ for current-state modeling and $25\%$ for future prediction. This approach enhances physical learning, enabling robust digital twins of complex deformable objects from sparse video data and reducing the gap between observed dynamics and physically plausible future states.

Abstract

In this paper, we aim to create physical digital twins of deformable objects under interaction. Existing methods focus more on the physical learning of current state modeling, but generalize worse to future prediction. This is because existing methods ignore the intrinsic physical properties of deformable objects, resulting in the limited physical learning in the current state modeling. To address this, we present NeuSpring, a neural spring field for the reconstruction and simulation of deformable objects from videos. Built upon spring-mass models for realistic physical simulation, our method consists of two major innovations: 1) a piecewise topology solution that efficiently models multi-region spring connection topologies using zero-order optimization, which considers the material heterogeneity of real-world objects. 2) a neural spring field that represents spring physical properties across different frames using a canonical coordinate-based neural network, which effectively leverages the spatial associativity of springs for physical learning. Experiments on real-world datasets demonstrate that our NeuSping achieves superior reconstruction and simulation performance for current state modeling and future prediction, with Chamfer distance improved by 20% and 25%, respectively.

Figures (4)

• Figure 1: Motivation of our NeuSpring. We show the reconstruction & resimulation and future prediction performance for PhysTwin jiang2025phystwin and our proposed method. We also visualize the approximated material based on the estimated physical property of springs--stiffness.
• Figure 2: Overview of Our NeuSpring.
• Figure 3: Qualitative Results on Reconstruction & Resimulation and Future Prediction.
• Figure 4: Effect of the Cluster Number.