Physics-Informed Learning of Characteristic Trajectories for Smoke Reconstruction

TL;DR

We address the problem of reconstructing 3D smoke flows and obstacle geometry from sparse RGB views by introducing the Neural Characteristic Trajectory Field (NCTF), a topology-free Eulerian–Lagrangian representation that implicitly models Lagrangian trajectories. The method encodes trajectory information with an encoder and decodes it to obtain temporally evolving positions, enabling a closed-form velocity integral and efficient flow-map evaluation between arbitrary frames. NCTF is integrated with a dual-density NeRF and SDF-based obstacle boundaries, using intrinsic (cycle and feature) and physical (NSE, transport) constraints to enforce long-term conservation while preserving short-term physics. Experiments on synthetic hybrid scenes and real ScalarFlow captures demonstrate improved obstacle geometry fidelity, reduced density–color ambiguities, and more accurate velocity and vorticity fields, highlighting the practical potential for high-fidelity fluid reconstruction in complex environments.

Abstract

We delve into the physics-informed neural reconstruction of smoke and obstacles through sparse-view RGB videos, tackling challenges arising from limited observation of complex dynamics. Existing physics-informed neural networks often emphasize short-term physics constraints, leaving the proper preservation of long-term conservation less explored. We introduce Neural Characteristic Trajectory Fields, a novel representation utilizing Eulerian neural fields to implicitly model Lagrangian fluid trajectories. This topology-free, auto-differentiable representation facilitates efficient flow map calculations between arbitrary frames as well as efficient velocity extraction via auto-differentiation. Consequently, it enables end-to-end supervision covering long-term conservation and short-term physics priors. Building on the representation, we propose physics-informed trajectory learning and integration into NeRF-based scene reconstruction. We enable advanced obstacle handling through self-supervised scene decomposition and seamless integrated boundary constraints. Our results showcase the ability to overcome challenges like occlusion uncertainty, density-color ambiguity, and static-dynamic entanglements. Code and sample tests are at \url{https://github.com/19reborn/PICT_smoke}.

Figures (16)

• Figure 1: The Neural Characteristic Trajectory Field comprises a Trajectory Encoder extracting consistent features along trajectories and a Trajectory Decoder for trajectory prediction and velocity extraction via auto-differentiation.
• Figure 2: Method overview. We model static objects using SDF and color fields, and dynamic smoke using a time-varying NeRF. The Neural Characteristic Trajectory Field enables end-to-end optimization of long-term mapping constraints and short-term physics priors. We utilized boundary conditions extracted from SDF-based obstacle geometry.
• Figure 3: Illustration of trajectory feature in the neural trajectory field. Features along specific trajectories remain consistent. We use the first three dimensions of the trajectory feature and normalize them based on the maximum and minimum values across sequences to visualize the color.
• Figure 4: Front-side-top illustrations of Lagrangian and NeRF density. The Lagrangian density yields more robust and smoother outcomes in the reconstructed density and its gradients, whereas the NeRF density contains richer high-frequency details.
• Figure 5: Without the proposed self-supervised scene decomposition constraints, static obstacles are likely to be modeled by the dynamic NeRF, leading to decomposition artifacts.