FlowCapX: Physics-Grounded Flow Capture with Long-Term Consistency

TL;DR

FlowCapX tackles the challenge of reconstructing physically consistent, high-fidelity velocity fields from sparse video data over long horizons. It introduces a two-level neural framework: a coarse level that enforces long-term physics via a set of vorticity- and transport-based losses, and a fine level that captures turbulent details with scale-appropriate regularization, with a final fusion that preserves global structure and fine features. Key contributions include a long-term transport loss, a velocity–vorticity formulation loss, kinetic-energy and boundary losses, and a warp/projection scheme for fine-scale advection, all validated by quantitative metrics and downstream tasks like tracer visualization and re-simulation. The approach yields state-of-the-art velocity reconstruction, enabling accurate flow analysis and more realistic visualizations, tracers, and simulations in sparse-view scenarios.

Abstract

We present FlowCapX, a physics-enhanced framework for flow reconstruction from sparse video inputs, addressing the challenge of jointly optimizing complex physical constraints and sparse observational data over long time horizons. Existing methods often struggle to capture turbulent motion while maintaining physical consistency, limiting reconstruction quality and downstream tasks. Focusing on velocity inference, our approach introduces a hybrid framework that strategically separates representation and supervision across spatial scales. At the coarse level, we resolve sparse-view ambiguities via a novel optimization strategy that aligns long-term observation with physics-grounded velocity fields. By emphasizing vorticity-based physical constraints, our method enhances physical fidelity and improves optimization stability. At the fine level, we prioritize observational fidelity to preserve critical turbulent structures. Extensive experiments demonstrate state-of-the-art velocity reconstruction, enabling velocity-aware downstream tasks, e.g., accurate flow analysis, scene augmentation with tracer visualization and re-simulation.

Figures (20)

• Figure 1: Method overview. We utilize two distinct neural networks to reconstruct the velocity field at coarse and fine levels. The coarse-level network emphasizes long-term physical consistency, while the fine-level network recovers observational details within the physically valid regions defined by the coarse level. Ultimately, we merge the two into a unified reconstruction that preserves both physical correctness and detailed turbulent motion.
• Figure 4: Visualization of re-simulation results on the Cylinder scene. Compared to PINF and PICT, our method achieves finer details and better alignment with the ground truth.
• Figure 5: Visualization of re-simulation results on the ScalarSyn scene. Our method better reproduces the fine high-frequency structures of smoke compared to PINF and PICT, while also avoiding the introduction of unphysical noise compared to HyFluid.
• Figure 7: Tracer visualization results on ScalarSyn. As shown in the visualization, all baseline methods—PINF, PICT, and HyFluid—incorrectly lift the paper pieces that should remain stationary. Moreover, HyFluid produces chaotic velocity fields that are physically implausible. In contrast, our method accurately reconstructs the motion, closely matching the ground truth.
• Figure 8: Ablation study of the kinetic loss $\mathcal{L}_{\text{kine}}$ and boundary loss $\mathcal{L}_{\text{bnd}}$ on the Cylinder scene. The kinetic loss $\mathcal{L}_{\text{kine}}$ promotes a clean reconstruction of the background velocity, while the boundary loss $\mathcal{L}_{\text{bnd}}$ enforces velocities at the boundaries to zero, ensuring that the boundary conditions are satisfied.