RTracker: Recoverable Tracking via PN Tree Structured Memory

TL;DR

This work tackles target loss in visual tracking by introducing recoverable tracking through joint tracker-detector operation. A Positive-Negative Tree Memory stores evolving positive and negative samples and employs walking rules with relative measurements, including the update $F_{new} = (F_x + F_{old} * N)/(N+1)$ and cosine similarity $S = \cos(F_x, F_{node})$, to assess target presence. Three control flows—normal, missing, and recovering—dynamically couple tracking and global detection to re-locate targets after disappearance. Experiments on VideoCube, LaSOT, LaSOT_ext, TNL2K, and GOT-10k show state-of-the-art performance and improved recovery ability, with ablations validating the PN-tree and walking rules.

Abstract

Existing tracking methods mainly focus on learning better target representation or developing more robust prediction models to improve tracking performance. While tracking performance has significantly improved, the target loss issue occurs frequently due to tracking failures, complete occlusion, or out-of-view situations. However, considerably less attention is paid to the self-recovery issue of tracking methods, which is crucial for practical applications. To this end, we propose a recoverable tracking framework, RTracker, that uses a tree-structured memory to dynamically associate a tracker and a detector to enable self-recovery ability. Specifically, we propose a Positive-Negative Tree-structured memory to chronologically store and maintain positive and negative target samples. Upon the PN tree memory, we develop corresponding walking rules for determining the state of the target and define a set of control flows to unite the tracker and the detector in different tracking scenarios. Our core idea is to use the support samples of positive and negative target categories to establish a relative distance-based criterion for a reliable assessment of target loss. The favorable performance in comparison against the state-of-the-art methods on numerous challenging benchmarks demonstrates the effectiveness of the proposed algorithm.

Figures (13)

• Figure 1: Performance on challenging sequences involving full occlusion, out-of-view, and viewpoint change. The proposed RTracker can accurately re-track the targets in these sequences after their reappearance.
• Figure 2: Definition of the PN tree. R, P, and N denote the root node, positive nodes, and negative nodes, respectively. New nodes are added to the tree through two ways: addition and merging. The walking paths are used for finding the support samples for target state determination.
• Figure 3: Overall flow of the proposed algorithm. Our proposed tracking method dynamically associates the tracker and detector based on the target state. It contains three associating processes: 1) normal case flow, which validates the target state normal, utilizing only the tracker for tracking; 2) target missing flow, which confirms the target as lost, activating the detector for global searching; 3) target recovering flow, which detecting until the lost target recovering, the detector stopped and reactivating the tracker.
• Figure 4: Evaluation of the recovery ability on LaSOT. The success rate is the percentage at which a tracker successfully recovers disappeared targets within specific frame numbers.
• Figure 5: Visualized results of the proposed algorithm, MixViT, OSTrack, and SeqTrack method on four challenging sequences with drastic changes. This indicates that our RTracker performs well with the support of detection through PN tree memory, whereas the other method that relies only on the tracker faces difficulties with these sequences.