TrackNetV5: Residual-Driven Spatio-Temporal Refinement and Motion Direction Decoupling for Fast Object Tracking

TL;DR

TrackNetV5 tackles the challenge of fast, small-object tracking in sports by introducing Motion Direction Decoupling (MDD) to recover directional motion priors and a Residual-Driven Spatio-Temporal Refinement (R-STR) head to refine coarse predictions using temporal context. The MDD encodes movement polarity into positive and negative traces, while R-STR uses a lightweight TSATTHead to estimate a residual correction on top of an initial draft, all within a coarse-to-fine, end-to-end pipeline that extends the TrackNetV2 backbone. The approach yields state-of-the-art F1-score (0.9859) and high accuracy (0.9733) on TrackNetV2 with only a 3.7% FLOPs increase, maintaining real-time性能. The method also demonstrates strong generalization on the LoveAll dataset, emphasizing robustness to lighting and viewpoint changes, and presents a practical balance between precision and recall for real-world sports analytics.

Abstract

The TrackNet series has established a strong baseline for fast-moving small object tracking in sports. However, existing iterations face significant limitations: V1-V3 struggle with occlusions due to a reliance on purely visual cues, while TrackNetV4, despite introducing motion inputs, suffers from directional ambiguity as its absolute difference method discards motion polarity. To overcome these bottlenecks, we propose TrackNetV5, a robust architecture integrating two novel mechanisms. First, to recover lost directional priors, we introduce the Motion Direction Decoupling (MDD) module. Unlike V4, MDD decomposes temporal dynamics into signed polarity fields, explicitly encoding both movement occurrence and trajectory direction. Second, we propose the Residual-Driven Spatio-Temporal Refinement (R-STR) head. Operating on a coarse-to-fine paradigm, this Transformer-based module leverages factorized spatio-temporal contexts to estimate a corrective residual, effectively recovering occluded targets. Extensive experiments on the TrackNetV2 dataset demonstrate that TrackNetV5 achieves a new state-of-the-art F1-score of 0.9859 and an accuracy of 0.9733, significantly outperforming previous versions. Notably, this performance leap is achieved with a marginal 3.7% increase in FLOPs compared to V4, maintaining real-time inference capabilities while delivering superior tracking precision.

Figures (12)

• Figure 1: Overall Architecture of TrackNetV5.
• Figure 2: Detailed architecture of the V2 Encoder-Decoder.
• Figure 3: The overall architecture of the Motion Direction Decoupling (MDD) mechanism.
• Figure 4: Schematic illustration of the Polarity Decomposition Layer.
• Figure 5: The workflow of the Residual-Driven Spatio-Temporal Refinement (R-STR) module. The Dropout mechanism is active only during the training phase.