Global Transport for Fluid Reconstruction with Learned Self-Supervision

TL;DR

This work tackles reconstructing volumetric fluid motion from highly under-constrained observations, potentially a single viewpoint. It introduces a global transport formulation that evolves the initial density $\rho^0$ through time via a differentiable transport operator, ensuring consistent transport across the full trajectory. Alearned visual prior via a discriminator (RaLSGAN) constrains unseen-view plausibility, while differentiable rendering with non-linear lighting and self-shadowing ties the physics to image formation. The method combines a Navier–Stokes–inspired physical prior, a set of transport losses, and a differentiable renderer in an end-to-end optimization, enabling realistic, temporally coherent fluid motion reconstructions from sparse views; it achieves state-of-the-art results on synthetic multi-view data and real single-view data, and demonstrates robust single-view reconstructions guided by learned self-supervision. The approach offers a practical path for physics-informed, data-efficient fluid reconstruction with potential applications in visualization, medical imaging, and computational fluid dynamics, albeit with limitations in handling dissipation and opaque obstacles.

Abstract

We propose a novel method to reconstruct volumetric flows from sparse views via a global transport formulation. Instead of obtaining the space-time function of the observations, we reconstruct its motion based on a single initial state. In addition we introduce a learned self-supervision that constrains observations from unseen angles. These visual constraints are coupled via the transport constraints and a differentiable rendering step to arrive at a robust end-to-end reconstruction algorithm. This makes the reconstruction of highly realistic flow motions possible, even from only a single input view. We show with a variety of synthetic and real flows that the proposed global reconstruction of the transport process yields an improved reconstruction of the fluid motion.

Figures (14)

• Figure 1: We propose a novel algorithm that reconstructs the motion $\mathrm{\mathbf{u}}^{0..F}$ of a single initial state $\rho^0$ over the full course of $F$ frames of an input sequence, i.e., its global transport. Based on a learned self-supervision, our algorithm yields a realistic motion for highly under-constrained scenarios such as a single input view.
• Figure 2: Multi-view evaluation with synthetic data: (a-e) Ablation with reference shown in (f). The different versions of the ablation (details in Section \ref{['sec:res.multi']}) continually improve density reconstruction and motion. (g-i) Comparison with previous work ScalarFlow eckertScalarFlowLargescaleVolumetric2019, TomoFluid zangTomoFluidReconstructingDynamic2020, and NeuralVolumes lombardiNeuralVolumesLearning2019. Our method in (e) yields an improved density reconstruction, in addition to a coherent and physical transport (see also Table \ref{['tab:eval.synth']}).
• Figure 3: Other methods for multi-view reconstruction (a,b) compared to our approach (c), versus the nearest target from a $30^{\circ}$ rotated viewpoint (d). Even for the new viewpoint, our reconstruction matches the structures of the target, while the other methods produce overly smooth reconstructions. The SF eckertScalarFlowLargescaleVolumetric2019 result is representative for TF zangTomoFluidReconstructingDynamic2020 here.
• Figure 4: Evaluation of transport accuracy for an initial state (a) advected with the velocity reconstructions of TF (b) and SF (c). Our result (d) is closest to the reference in (e).
• Figure 5: Single-view reconstructions of a captured fluid (target at $0^{\circ}$). Top & middle: density from a $90^{\circ}$ side view. Bottom: rendered visualization at $60^{\circ}$. Our version with discriminator (d) yields coherent and sharp flow structures, preventing any smearing along the unconstrained direction (a, b) or streak artefacts (c). Temporal supervision xieTempoGANTemporallyCoherent2018 has no added benefit (e), while the purely learning-based method henzlerEscapingPlatoCave2019 fails to generate coherent features (f).