Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PhysConvex: Physics-Informed 3D Dynamic Convex Radiance Fields for Reconstruction and Simulation

Dan Wang, Xinrui Cui, Serge Belongie, Ravi Ramamoorthi

TL;DR

PhysConvex is proposed, a Physics-informed 3D Dynamic Convex Radiance Field that unifies visual rendering and physical simulation, and achieves high-fidelity reconstruction of geometry, appearance, and physical properties from videos, outperforming existing methods.

Abstract

Reconstructing and simulating dynamic 3D scenes with both visual realism and physical consistency remains a fundamental challenge. Existing neural representations, such as NeRFs and 3DGS, excel in appearance reconstruction but struggle to capture complex material deformation and dynamics. We propose PhysConvex, a Physics-informed 3D Dynamic Convex Radiance Field that unifies visual rendering and physical simulation. PhysConvex represents deformable radiance fields using physically grounded convex primitives governed by continuum mechanics. We introduce a boundary-driven dynamic convex representation that models deformation through vertex and surface dynamics, capturing spatially adaptive, non-uniform deformation, and evolving boundaries. To efficiently simulate complex geometries and heterogeneous materials, we further develop a reduced-order convex simulation that advects dynamic convex fields using neural skinning eigenmodes as shape- and material-aware deformation bases with time-varying reduced DOFs under Newtonian dynamics. Convex dynamics also offers compact, gap-free volumetric coverage, enhancing both geometric efficiency and simulation fidelity. Experiments demonstrate that PhysConvex achieves high-fidelity reconstruction of geometry, appearance, and physical properties from videos, outperforming existing methods.

PhysConvex: Physics-Informed 3D Dynamic Convex Radiance Fields for Reconstruction and Simulation

TL;DR

PhysConvex is proposed, a Physics-informed 3D Dynamic Convex Radiance Field that unifies visual rendering and physical simulation, and achieves high-fidelity reconstruction of geometry, appearance, and physical properties from videos, outperforming existing methods.

Abstract

Reconstructing and simulating dynamic 3D scenes with both visual realism and physical consistency remains a fundamental challenge. Existing neural representations, such as NeRFs and 3DGS, excel in appearance reconstruction but struggle to capture complex material deformation and dynamics. We propose PhysConvex, a Physics-informed 3D Dynamic Convex Radiance Field that unifies visual rendering and physical simulation. PhysConvex represents deformable radiance fields using physically grounded convex primitives governed by continuum mechanics. We introduce a boundary-driven dynamic convex representation that models deformation through vertex and surface dynamics, capturing spatially adaptive, non-uniform deformation, and evolving boundaries. To efficiently simulate complex geometries and heterogeneous materials, we further develop a reduced-order convex simulation that advects dynamic convex fields using neural skinning eigenmodes as shape- and material-aware deformation bases with time-varying reduced DOFs under Newtonian dynamics. Convex dynamics also offers compact, gap-free volumetric coverage, enhancing both geometric efficiency and simulation fidelity. Experiments demonstrate that PhysConvex achieves high-fidelity reconstruction of geometry, appearance, and physical properties from videos, outperforming existing methods.
Paper Structure (24 sections, 10 equations, 6 figures, 6 tables)

This paper contains 24 sections, 10 equations, 6 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Physics-Based Simulation.
  4. Video-based Physical Reconstruction and Simulation.
  5. Preliminaries
  6. Continuum Mechanics.
  7. Model Order Reduction.
  8. 3D Convex Splatting.
  9. Method
  10. Boundary-driven Dynamic Convex Fields
  11. Physical Reduced-order Convex Simulation
  12. Convex Vertex Dynamics.
  13. Convex Surface Dynamics.
  14. Differentiable Rendering and Simulation
  15. Appearance and Geometry Optimization.
  16. ...and 9 more sections

Figures (6)

  • Figure 1: PhysConvex introduces boundary-driven dynamic convex fields integrated with reduced-order convex simulation for unified 3D reconstruction and dynamics. It recovers appearance, geometry, and physics from multi-view videos, enabling mesh-free, physically consistent, visually realistic simulation.
  • Figure 2: Center-driven Gaussian dynamics have limited flexibility for non-uniform deformation and evolving boundaries, while boundary-driven convex dynamics enable spatially adaptive vertex advection, explicit surface evolution, and polyhedral structural refinement.
  • Figure 3: PhysConvex jointly reconstructs the geometry, appearance, and physical properties of dynamic objects by integrating boundary-driven dynamic convex field, differentiable reduced-order convex simulation, and rendering. Training proceeds in two stages: (1) reconstructing an undeformed convex field from the first multi-view frame, and (2) advecting the boundary-driven convex field via reduced-order simulation while jointly optimizing physical properties via differentiable simulation and rendering under single-view video supervision.
  • Figure 4: Comparison with baseline li2023paccai2024giczhong2024springgauschen2025vid2sim on dynamic reconstruction and future state prediction. Our method preserves high-quality geometry and appearance while producing physically plausible dynamics.
  • Figure 5: Generalization to novel materials. Elastic (stiff): $E{=}10^7$, $\nu{=}0.49$. Elastic (soft): $E = 8000$, $\nu = 0.4$. Plasticine: $\tau_Y{=}500$. Sand: $\theta_f{=}10^\circ$.
  • ...and 1 more figures