vSTMD: Visual Motion Detection for Extremely Tiny Target at Various Velocities

TL;DR

This work tackles the challenge of visual motion detection for extremely tiny targets across varied velocities by proposing vSTMD, a learning-free, bio-inspired model. The core innovations are cross-Inhibition Dynamic Potential (cIDP) for self-adaptive motion integration and Collaborative Directional Gradient Calculation (CDGC) for efficient, robust orientation estimation. On the RIST dataset, vSTMD and its feedback-enhanced variant vSTMD-F achieve substantial improvements over state-of-the-art STMD approaches in localization (mF1) and orientation (AAE), while maintaining real-time performance (over 560 FPS). The approach also generalizes to cross-task scenarios (XS-VID) and maritime search-and-rescue tasks, offering a fast, interpretable alternative to data-driven methods with broad practical impact for dynamic, cluttered environments.

Abstract

Visual motion detection for extremely tiny (ET-) targets is challenging, due to their category-independent nature and the scarcity of visual cues, which often incapacitate mainstream feature-based models. Natural architectures with rich interpretability offer a promising alternative, where STMD architectures derived from insect visual STMD (Small Target Motion Detector) pathways have demonstrated their effectiveness. However, previous STMD models are constrained to a narrow velocity range, hindering their efficacy in real-world scenarios where targets exhibit diverse and unstable dynamics. To address this limitation, we present vSTMD, a learning-free model for motion detection of ET-targets at various velocities. Our key innovations include: (1) a cross-Inhibition Dynamic Potential (cIDP) that serves as a self-adaptive mechanism efficiently capturing motion cues across a wide velocity spectrum, and (2) the first Collaborative Directional Gradient Calculation (CDGC) strategy, which enhances orienting accuracy and robustness while reducing computational overhead to one-eighth of previously isolated strategies. Evaluated on the real-world dataset RIST, the proposed vSTMD and its feedback-facilitated variant vSTMD-F achieve relative $F_{1}$ gains of $30\%$ and $58\%$ over state-of-the-art (SOTA) STMD approaches, respectively. Furthermore, both models demonstrate competitive orientation estimation performance compared to SOTA deep learning-driven methods. Experiments also reveal the superiority of the natural architecture for ET-object motion detection - vSTMD is $60\times$ faster than contemporary data-driven methods, making it highly suitable for real-time applications in dynamic scenarios and complex backgrounds. Code is available at https://github.com/MingshuoXu/vSTMD.

Figures (8)

• Figure 1: Challenges in motion detection of extremely tiny (ET-) targets: sparse visual cues, unstable outline, and background dynamics in a cluttered scenario. The figure illustrates a distance bird flying against a dynamic and cluttered background across time $t_{1}$, $t_{2}$, and $t_{3}$, with the bird highlighted and zoomed in in red boxes for better visibility. The bird occupies only few dozen pixels, like a spot, and its outline varies by flapping wings and flying attitude, making it difficult for motion detection.
• Figure 2: Schematic of the proposed work. (a) Overall architecture, comprising sequential stages of pre-processing, motion integration, localization, and orientation estimation. vSTMD refers to the baseline model, while vSTMD-F denotes the variant incorporating the feedback pathway Wang2021time. (b) Detailed structure of the cross-Inhibition Dynamic Potential (cIDP) mechanism, where red double-line arrows indicate cross-inhibition pathways and DP represents the dynamic potential components. (c) The Collaborative Directional Gradient Calculation (CDGC) strategy employed for orientation estimation. In the schematic, circular elements represent pixel-wise operators, and warped cubes denote intermediate spatial feature maps that maintain the input image dimensions.
• Figure 3: Output comparison between the conventional delay-and-correlate mechanism (left column) and the proposed model (right column) at velocities ranging from 0.5 pixel per frame in the top row to 3 pixel per frame in the bottom row. The $L^{+}$ and $L^{-}$ signals denote the ON and OFF motion edges produced by extremely tiny targets. In the left column, the delay-and-correlate mechanism imposes a fixed temporal delay between the ON and OFF channels to ensure temporal correct correlation. As velocity increases, the delayed OFF signal becomes increasingly misaligned with the ON signal. Such misregistration shifts the response away from the target - outside the summits of ON and OFF motion edges, and reduces the response's amplitude. In contrast, the right column illustrates the proposed model’s self-adaptation across various velocities. The dynamic potentials of the proposed cIDP, $V^{+}$ and $V^{-}$, calculated by Formula (\ref{['Formula:ON_current']}) and (\ref{['Formula:OFF_current']}), steadily exhibit opposite gradients within targets, and their product yields robust responses across velocities and remains consistently localized within the actual target.
• Figure 4: The temporal signal processing schematic for motion localization. Panel (A) showcases a signal composed of a tiny target against a cluttered and dynamic background, recorded for a fixed pixel along the temporal axis from a set of images. In (B), temporal changes of the input signal are extracted and then separated into $L^{+}$ and $L^{-}$ based on brightness increases and decreases. Next, panel (C) illustrates the output of the cross-Inhibition Dynamic Potentials (cIDP) mechanism of ON ($V^{+}$) and OFF ($V^{-}$) with ipsilateral excitatory and contralateral inhibition junctions. Finally, these two potentials are multiplied to locate the motion of ET-targets in (D), while there is no response to the dynamic background or larger objects such as the tree.
• Figure 5: Robustness, accuracy, and efficiency of the proposed CDGC compared with traditional Isolated-DGC Wang2020DSTMD. (A) illustrates the robustness of the proposed CDGC (green arrows) by only leveraging front and rear motion edges (green hollow circle). In comparison, the Isolated-DGC (pink arrows) calculates correlations between a current motion edge and a previous motion edge outside the current target (solid pink circle), making it susceptible to interference from background motion. (B) shows the high directional accuracy of the proposed CDGC. (C) highlights the efficiency of the proposed CDGC over the conventional Isolated-DGC. For simplicity, we illustrate a central location and eight surrounding locations, along with how directional information can be calculated, where the neuron names are indicated in the upper left corner. In the proposed CDGC, direction information can be directly read out from the sharing gradient without additional computation. Consequently, the proposed CDGC requires only one correlation per spatial location, while the traditional Isolated-DGC necessitates eight correlations between ON and OFF signals at each spatial location.