Deep-learning Optical Flow Outperforms PIV in Obtaining Velocity Fields from Active Nematics

TL;DR

This work demonstrates that a RAFT-based deep-learning optical flow approach (DLOF) can surpass traditional PIV in measuring velocity fields in MT-based active nematics, especially at high filament densities. By leveraging a correlation pyramid and ConvGRU-driven refinement, DLOF delivers higher-resolution, smoother flow fields and better ground-truth agreement across both sparse MT labeling and bead-tracking ground truths. The results show substantial improvements in velocity accuracy, particularly for motions parallel to the nematic director, where PIV struggles due to uniform intensity contrasts. The findings position DLOF as a versatile, scalable tool for quantitative flow measurement in active matter and related soft-biophysical systems.

Abstract

Deep learning-based optical flow (DLOF) extracts features in adjacent video frames with deep convolutional neural networks. It uses those features to estimate the inter-frame motions of objects at the pixel level. In this article, we evaluate the ability of optical flow to quantify the spontaneous flows of MT-based active nematics under different labeling conditions. We compare DLOF against the commonly used technique, particle imaging velocimetry (PIV). We obtain flow velocity ground truths either by performing semi-automated particle tracking on samples with sparsely labeled filaments, or from passive tracer beads. We find that DLOF produces significantly more accurate velocity fields than PIV for densely labeled samples. We show that the breakdown of PIV arises because the algorithm cannot reliably distinguish contrast variations at high densities, particularly in directions parallel to the nematic director. DLOF overcomes this limitation. For sparsely labeled samples, DLOF and PIV produce results with similar accuracy, but DLOF gives higher-resolution fields. Our work establishes DLOF as a versatile tool for measuring fluid flows in a broad class of active, soft, and biophysical systems.

Figures (9)

• Figure 1: Microtubule (MT)-based active nematics. (A) Microscopic components of the active nematic liquid crystal. Kinesin motor clusters consume energy to actively slide neighboring MTs against each other. (B) The active nematic exhibits the spontaneous flow that deforms the nematic texture over time. All MTs are fluorescently labeled at 647 nm. Increased local intensity indicates a higher local filament concentration. The time step is 7.5 s. (C) In Experiment 1, the fully labeled MTs (top panel) are mixed with a sparse population of MTs that fluoresce at 488 nm (bottom panel), which are used to generate ground-truth velocity points. (D) In Experiment 2, the fully labeled MTs (top panel) are mixed with passivated microbeads, which are used to generate the ground-truth velocities (bottom panel).
• Figure 2: Main components of the DLOF model. (A) Feature extraction and construction of feature-level correlations: A convolutional neural network (CNN) is used to extract $D$ feature maps of resolution of $H \times W$ for each of the input images. Taking the inner product of the features maps of two images produces all-pair feature-level correlation volumes $\mathbf{C^1}$ of dimension $H \times W \times H \times W$. (B) Correlation pyramid: Multi-scale feature correlations are constructed by pooling the last two dimensions of $\mathbf{C^1}$, such that those dimensions are reduced by 1/2, 1/4, and 1/8, resulting in $\mathbf{C^2}$, $\mathbf{C^3}$, and $\mathbf{C^4}$, respectively. The first two dimensions preserve high-resolution information while multi-scale correlations enable the model to capture the motions of small fast-moving objects. (C) Correlation lookup for a pixel $\mathbf{x}$ in $I_1$: An estimate of the location of the correspondence $\mathbf{x'}$ (in $I_2$) is initialized by displacing $\mathbf{x}$ using the current flow estimate $\mathbf{f}$. The model then looks for the most correlated features in a neighborhood $\mathcal{N}(\mathbf{x'})_r$ centered at $\mathbf{x'}$ ($r=3$ in the figure), where all locations within $\mathcal{N}(\mathbf{x'})_r$ are used to index from the correlation pyramid $\{\mathbf{C^1}, \mathbf{C^2}, \mathbf{C^3}, \mathbf{C^4}\}$ to produce correlation features at all levels, which are further concatenated to form a single correlation feature map for the pixel $\mathbf{x}$ in $I_1$.
• Figure 3: DLOF outperforms PIV for densely labeled samples. (left) The trajectory of an individual MT, which is imaged every 1.5 seconds. MT true velocities (cyan arrows) are obtained by particle tracking. The velocity vectors estimated by PIV and DLOFs are indicated respectively with green and orange arrows. The insets depict the densely labeled MTs in local neighborhoods of the tracked labels at the indicated times. The high densities of the labels in the images pose a significant challenge to PIV, resulting in inaccurate velocity estimates. In contrast, DLOF produces highly accurate velocities. Particle tracking was extracted from a simultaneously imaged sparsely labeled channel.
• Figure 4: Comparing PIV and DLOF to single-filament tracking. Distribution of errors when comparing PIV and DLOF velocity fields from sparsely and densely labeled samples to single-filament tracking. The distributions of errors in the magnitude and orientation of the velocity (defined in the text) for PIV and DLOF. Errors are computed by comparing different estimates with particle tracking results. The mean relative speed errors for PIV are 42% and 19% for densely and sparsely labeled systems; errors for DLOF are 29% and 23%. The mean orientation errors for PIV are 44 degrees and 14 degrees for densely and sparsely labeled systems; errors for DLOF are 29 degrees and 17 degrees. The distributions are obtained from 4738 traced labels across 44 frames in Experiment 1.
• Figure 5: The improvement of DLOF over PIV increases as the velocity becomes parallel to the director field (for dense labels). Average relative speed error (A) and average orientation error (B) of PIV and DLOF as a function of the angle between ground truth velocity and director. PIV particularly breaks when the velocities are tangent to the MT bundles due to the uniform contrast of the dense labels along MT bundles. DLOF can handle the uniform contrast along MT bundles and thus produces much more accurate velocities.