PhysGaia: A Physics-Aware Benchmark with Multi-Body Interactions for Dynamic Novel View Synthesis

Abstract

We introduce PhysGaia, a novel physics-aware benchmark for Dynamic Novel View Synthesis (DyNVS) that encompasses both structured objects and unstructured physical phenomena. While existing datasets primarily focus on photorealistic appearance, PhysGaia is specifically designed to support physics-consistent dynamic reconstruction. Our benchmark features complex scenarios with rich multi-body interactions, where objects realistically collide and exchange forces. Furthermore, it incorporates a diverse range of materials, including liquid, gas, textile, and rheological substance, moving beyond the rigid-body assumptions prevalent in prior work. To ensure physical fidelity, all scenes in PhysGaia are generated using material-specific physics solvers that strictly adhere to fundamental physical laws. We provide comprehensive ground-truth information, including 3D particle trajectories and physical parameters (e.g., viscosity), enabling the quantitative evaluation of physical modeling. To facilitate research adoption, we also provide integration pipelines for recent 4D Gaussian Splatting models along with our dataset and their results. By addressing the critical shortage of physics-aware benchmarks, PhysGaia can significantly advance research in dynamic view synthesis, physics-based scene understanding, and the integration of deep learning with physical simulation, ultimately enabling more faithful reconstruction and interpretation of complex dynamic scenes.

Figures (13)

• Figure 1: Visualization of key properties of PhysGaia. Unlike existing benchmarks limited to single objects or materials, PhysGaia is a physics-aware benchmark featuring complex multi-body interactions across diverse substances (liquids, gases, rheological materials, and textiles). By providing ground-truth 3D trajectories and physical parameters, it uniquely enables the evaluation of physical realism alongside traditional photorealism. In addition to multi-body collisions, our dataset captures (c) splashing effects characterized by non-local rigid motion, as well as (d--e) complex optical phenomena such as specular reflection and refraction.
• Figure 2: Examples from the proposed physics-aware benchmark, PhysGaia. They exhibit complex physical interactions between multiple objects composed of diverse materials such as liquid, gas, rheological substance, and textile. More importantly, our benchmark enables evaluation of physics realism to foster physics reasoning in dynamic scenes.
• Figure 3: Examples of diverse modalities that users can generate from the provided simulation node graphs. This can facilitate adaptation to specific downstream tasks.
• Figure 4: Limitations of existing datasets. While ScalarFlow, PAC-NeRF, and Spring-Gaus address physical phenomena, they are limited in narrow coverage of physical materials, overly simplified dynamics, and an absence of rich multi-object interactions.
• Figure 5: Qualitative results of recent DyNVS methods on the jelly party scene with monocular setup. All methods struggle to accurately capture multi-body interactions by frequently exhibiting needle-like artifacts and failing to reconstruct dynamic elements accurately. Please refer to our supplementary documents for more qualitative results.