Dual-frame Fluid Motion Estimation with Test-time Optimization and Zero-divergence Loss

TL;DR

A new method is introduced that is completely self-supervised and notably outperforms its fully-supervised counterparts while requiring only 1% of the training samples used by previous methods, and features a novel zero-divergence loss that is specific to the domain of turbulent flow.

Abstract

3D particle tracking velocimetry (PTV) is a key technique for analyzing turbulent flow, one of the most challenging computational problems of our century. At the core of 3D PTV is the dual-frame fluid motion estimation algorithm, which tracks particles across two consecutive frames. Recently, deep learning-based methods have achieved impressive accuracy in dual-frame fluid motion estimation; however, they heavily depend on large volumes of labeled data. In this paper, we introduce a new method that is completely self-supervised and notably outperforms its fully-supervised counterparts while requiring only 1% of the training samples (without labels) used by previous methods. Our method features a novel zero-divergence loss that is specific to the domain of turbulent flow. Inspired by the success of splat operation in high-dimensional filtering and random fields, we propose a splat-based implementation for this loss which is both efficient and effective. The self-supervised nature of our method naturally supports test-time optimization, leading to the development of a tailored Dynamic Velocimetry Enhancer (DVE) module. We demonstrate that strong cross-domain robustness is achieved through test-time optimization on unseen leave-one-out synthetic domains and real physical/biological domains. Code, data and models are available at https://github.com/Forrest-110/FluidMotionNet.

Figures (9)

• Figure 1: Paradigm Shift: Given two frames of flow particles $X_t$ and $X_{t+\Delta t}$, DeepPTV liang2021deepptv adopts a two-stage network for large- and small-scale motion refinement. GotFlow3D liang2023recurrent trains a correspondence learning network and an RNN-based residual prediction network. They are trained in a fully supervised manner with annotated data and do not support test-time optimization. Our purely self-supervised method diverges from these approaches and employs DVE (see Sec. \ref{['method-test']}) for on-the-fly test-time optimization. The "Snowflake" denotes frozen weights.
• Figure 2: Upper: Training Phase. First, we (a) use input point clouds to construct graphs, which are then passed through a trainable (b) feature extractor, and we solve a (c) optimal transport problem using self-supervised loss terms including (g) reconstruction loss, (f) smooth loss, and (e) zero-divergence loss for initial flow estimation. Lower: (h) Test-Time DVE. With the initial flow estimate $\mathbf{F}_{\rm init}$, we optimize a residual $\mathbf{R}$ to generate the final flow $\mathbf{F}$ using another reconstruction loss (g*).
• Figure 3: LEFT: A visualization of fluid flow in Fluidflow3D data. RIGHT: The divergence loss in our training phase is obtained by splatting the original sparse flow to grid points and then minimizing the divergence loss on the resulting grid points.
• Figure 4: (Top) Benchmarking Against Fully Supervised Methods. $P_{\text{train}}$ signifies the count of trainable parameters. $T_{\text{test}}$ stands for inference time for each sample. The best results are marked in bold. (Bottom) Performance Across Flow Cases. The best results are marked in bold, with the runners-up underlined. The subplots on the right visualize these three cases. The warmer color indicates a higher flow speed. All models are trained on full data, except Ours (1%).
• Figure 5: (a) Leave-one-out domain EPE Comparison: "Flow Cases" stands for the flow case we test on with the model trained on the rest five cases. (b) Comparison of EPE with Limited Training Data. (c) Performance Drop related to Limited Training Data. The Y-axis shows the major matric EPE, and the X-axis indicates the percentage of the training dataset utilized.