Estimating Dynamic Flow Features in Groups of Tracked Objects

TL;DR

The paper tackles the challenge of extracting dynamical flow information from sequences of images containing arbitrary groups of tracked objects. It merges deep object detection/tracking with Lagrangian Gradient Regression (LGR) to estimate velocity-field gradients from sparse trajectories and to compute objective metrics such as FTLE, LAVD, and vorticity deviation. The authors present a modular three-stage pipeline (detection, tracking, gradient-based structure identification) and validate it on a laboratory debris-flow case and a real-field pond debris scenario, showing robust gradient estimation where conventional methods fail. The approach enables gradient-based analyses in diverse, feature-rich contexts using affordable hardware, and it opens the door to multi-class dynamical studies and broader applications in swarms, traffic, microfluidics, and environmental flows.

Abstract

Interpreting motion captured in image sequences is crucial for a wide range of computer vision applications. Typical estimation approaches include optical flow (OF), which approximates the apparent motion instantaneously in a scene, and multiple object tracking (MOT), which tracks the motion of subjects over time. Often, the motion of objects in a scene is governed by some underlying dynamical system which could be inferred by analyzing the motion of groups of objects. Standard motion analyses, however, are not designed to intuit flow dynamics from trajectory data, making such measurements difficult in practice. The goal of this work is to extend gradient-based dynamical systems analyses to real-world applications characterized by complex, feature-rich image sequences with imperfect tracers. The tracer trajectories are tracked using deep vision networks and gradients are approximated using Lagrangian gradient regression (LGR), a tool designed to estimate spatial gradients from sparse data. From gradients, dynamical features such as regions of coherent rotation and transport barriers are identified. The proposed approach is affordably implemented and enables advanced studies including the motion analysis of two distinct object classes in a single image sequence. Two examples of the method are presented on data sets for which standard gradient-based analyses do not apply.

Figures (6)

• Figure 1: Schematic of the experiment used in the case study.
• Figure 2: Computing flow gradients from an image sequence using standard procedures from experimental fluid mechanics (bottom left) and from computer vision (bottom right). The algorithm used on the left is multi-pass PIV Raffel.Kompenhans_PIVbook_2018, and the algorithm used on the right is RAFT TeedDeng_RAFT_2020. The results display vorticity ($\partial v/ \partial x - \partial u/ \partial y$). From them it is clear that existing methods are not suited for spatial flow gradient estimation. In both cases, there is significant noise and the existence of many spurious features.
• Figure 3: Sample results from the detection and tracking stages of the case study. (a) A sample frame displaying detections using windowing with 10% overlap. Color indicates the windowed sub-image where detections were made. Rods overlapping the boundaries of windows have not yet been corrected, but will be in further steps. (b) A subset of trajectories length 50 or longer from the small birch spheres dataset. All trajectories start in $x/t \in [-1, 1.25]$. Color is determined by $y/t$. If the first instance in a trajectory is in $y/t \in [-0.5, 0.5]$ it is colored black.
• Figure 4: Results of the proposed analysis on the debris flow data. Top: Small birch spheres as tracers. Bottom: All birch debris as tracers. Vorticity is computed in (a) and (d), where red indicates positive value (counterclockwise rotation) and blue indicates negative value. Finite-time stretching (FTLE) is presented in (b) and (e), where yellow indicates large value and white is zero, and finite-time rotation (LAVD) is given in (c) and (f), where bright blue indicates large values.
• Figure 5: An example image from the turtle ponds data set being analyzed in section \ref{['sec:field test: pond debris']}. Image features which complicate normal motion and gradient estimates are highlighted.