Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Hamiltonian Deformation Fields for Dynamic Scene Rendering

Hai-Long Qin, Sixian Wang, Guo Lu, Jincheng Dai

TL;DR

NeHaD addresses the challenge of physically plausible dynamic scene rendering by embedding Hamiltonian dynamics into neural Gaussian deformation fields. It replaces conventional MLP decoders with Hamiltonian neural networks, employs Boltzmann equilibrium decomposition to separate static and dynamic Gaussians, and enforces physics-informed constraints including second-order symplectic integration and local rigidity regularization. The method further extends to streaming via scale-aware mipmapping and progressive optimization. Across synthetic and real-world dynamic scenes, NeHaD demonstrates improved rendering realism and motion coherence while maintaining practical rendering speeds, marking the first application of Hamiltonian mechanics to neural Gaussian deformation. This framework offers a principled path toward robust, real-time dynamic view synthesis with physics-guided guarantees.

Abstract

Representing and rendering dynamic scenes with complex motions remains challenging in computer vision and graphics. Recent dynamic view synthesis methods achieve high-quality rendering but often produce physically implausible motions. We introduce NeHaD, a neural deformation field for dynamic Gaussian Splatting governed by Hamiltonian mechanics. Our key observation is that existing methods using MLPs to predict deformation fields introduce inevitable biases, resulting in unnatural dynamics. By incorporating physics priors, we achieve robust and realistic dynamic scene rendering. Hamiltonian mechanics provides an ideal framework for modeling Gaussian deformation fields due to their shared phase-space structure, where primitives evolve along energy-conserving trajectories. We employ Hamiltonian neural networks to implicitly learn underlying physical laws governing deformation. Meanwhile, we introduce Boltzmann equilibrium decomposition, an energy-aware mechanism that adaptively separates static and dynamic Gaussians based on their spatial-temporal energy states for flexible rendering. To handle real-world dissipation, we employ second-order symplectic integration and local rigidity regularization as physics-informed constraints for robust dynamics modeling. Additionally, we extend NeHaD to adaptive streaming through scale-aware mipmapping and progressive optimization. Extensive experiments demonstrate that NeHaD achieves physically plausible results with a rendering quality-efficiency trade-off. To our knowledge, this is the first exploration leveraging Hamiltonian mechanics for neural Gaussian deformation, enabling physically realistic dynamic scene rendering with streaming capabilities.

Neural Hamiltonian Deformation Fields for Dynamic Scene Rendering

TL;DR

NeHaD addresses the challenge of physically plausible dynamic scene rendering by embedding Hamiltonian dynamics into neural Gaussian deformation fields. It replaces conventional MLP decoders with Hamiltonian neural networks, employs Boltzmann equilibrium decomposition to separate static and dynamic Gaussians, and enforces physics-informed constraints including second-order symplectic integration and local rigidity regularization. The method further extends to streaming via scale-aware mipmapping and progressive optimization. Across synthetic and real-world dynamic scenes, NeHaD demonstrates improved rendering realism and motion coherence while maintaining practical rendering speeds, marking the first application of Hamiltonian mechanics to neural Gaussian deformation. This framework offers a principled path toward robust, real-time dynamic view synthesis with physics-guided guarantees.

Abstract

Representing and rendering dynamic scenes with complex motions remains challenging in computer vision and graphics. Recent dynamic view synthesis methods achieve high-quality rendering but often produce physically implausible motions. We introduce NeHaD, a neural deformation field for dynamic Gaussian Splatting governed by Hamiltonian mechanics. Our key observation is that existing methods using MLPs to predict deformation fields introduce inevitable biases, resulting in unnatural dynamics. By incorporating physics priors, we achieve robust and realistic dynamic scene rendering. Hamiltonian mechanics provides an ideal framework for modeling Gaussian deformation fields due to their shared phase-space structure, where primitives evolve along energy-conserving trajectories. We employ Hamiltonian neural networks to implicitly learn underlying physical laws governing deformation. Meanwhile, we introduce Boltzmann equilibrium decomposition, an energy-aware mechanism that adaptively separates static and dynamic Gaussians based on their spatial-temporal energy states for flexible rendering. To handle real-world dissipation, we employ second-order symplectic integration and local rigidity regularization as physics-informed constraints for robust dynamics modeling. Additionally, we extend NeHaD to adaptive streaming through scale-aware mipmapping and progressive optimization. Extensive experiments demonstrate that NeHaD achieves physically plausible results with a rendering quality-efficiency trade-off. To our knowledge, this is the first exploration leveraging Hamiltonian mechanics for neural Gaussian deformation, enabling physically realistic dynamic scene rendering with streaming capabilities.

Paper Structure

This paper contains 23 sections, 1 theorem, 39 equations, 8 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. NeRF-based Dynamic Scene Rendering
  4. Gaussian-based Dynamic Scene Rendering
  5. Preliminaries
  6. Gaussian Splatting
  7. Hamiltonian Mechanics
  8. Methodology
  9. Neural Hamiltonian Deformation Fields
  10. Boltzmann Equilibrium Decomposition
  11. Physics-Informed Constraints
  12. Experiments
  13. Experimental Setup
  14. Applications: Streaming
  15. Experimental Results
  16. ...and 8 more sections

Key Result

theorem 1

Let $\boldsymbol{F}: \mathbb{R}^3 \to \mathbb{R}^3$ be a sufficiently smooth vector field that decays rapidly at infinity. Then $\boldsymbol{F}$ admits a unique decomposition: where $\boldsymbol{F}_{\text{conservative}}$ is irrotational ($\nabla \times \boldsymbol{F}_{\text{conservative}} = \boldsymbol{0}$) and $\boldsymbol{F}_{\text{solenoidal}}$ is divergence-free ($\nabla \cdot \boldsymbol{F}_

Figures (8)

  • Figure 1: Overall pipeline of NeHaD. (from left to right) An HNN with MLP baseline learns conservation laws from data. Through backpropagation of Hamiltonian gradients, the HNN optimizes vector fields and predicts Gaussian deformations (position, scaling, rotation) via adapters. The Boltzmann equilibrium decomposition decides which primitives should not be deformed with soft masks, i.e., smaller deviations from equilibrium maintain static during deformation. Physics-informed constraints including symplectic integration and rigidity regularization are used to preserve system properties.
  • Figure 2: Qualitative results of adaptive streaming. NeHaD adapts effectively to progressive streaming across varying allocated rates.
  • Figure 3: Ablation visualization. Without the proposed modules, rendering results exhibit motion artifacts and visual distortions. In contrast, our complete NeHaD model produces significantly higher quality results, demonstrating the effectiveness of our approach.
  • Figure 4: Visualization of Gaussian spatial distributions and motion trajectories during deformation. Compared to baseline 4dgs using MLP-based decoders, our HNN-based approach inherently respects Hamiltonian mechanics principles, resulting in more directed, ordered, and natural movements instead of relying solely on stochastic optimization.
  • Figure 5: Qualitative results on D-NeRF dnerf dataset (N/A: not available).
  • ...and 3 more figures

Theorems & Definitions (2)

  • theorem 1: Helmholtz Decomposition Theorem
  • proof