Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OneTracker: Unifying Visual Object Tracking with Foundation Models and Efficient Tuning

Lingyi Hong, Shilin Yan, Renrui Zhang, Wanyun Li, Xinyu Zhou, Pinxue Guo, Kaixun Jiang, Yiting Chen, Jinglun Li, Zhaoyu Chen, Wenqiang Zhang

TL;DR

This work presents a general framework to unify various tracking tasks, termed as One Tracker, which is consisted of Foundation Tracker and Prompt Tracker, and conducts extensive experiments on 6 popular tracking tasks across 11 benchmarks and the One- Tracker outperforms other models and achieves state-of-the-art performance.

Abstract

Visual object tracking aims to localize the target object of each frame based on its initial appearance in the first frame. Depending on the input modility, tracking tasks can be divided into RGB tracking and RGB+X (e.g. RGB+N, and RGB+D) tracking. Despite the different input modalities, the core aspect of tracking is the temporal matching. Based on this common ground, we present a general framework to unify various tracking tasks, termed as OneTracker. OneTracker first performs a large-scale pre-training on a RGB tracker called Foundation Tracker. This pretraining phase equips the Foundation Tracker with a stable ability to estimate the location of the target object. Then we regard other modality information as prompt and build Prompt Tracker upon Foundation Tracker. Through freezing the Foundation Tracker and only adjusting some additional trainable parameters, Prompt Tracker inhibits the strong localization ability from Foundation Tracker and achieves parameter-efficient finetuning on downstream RGB+X tracking tasks. To evaluate the effectiveness of our general framework OneTracker, which is consisted of Foundation Tracker and Prompt Tracker, we conduct extensive experiments on 6 popular tracking tasks across 11 benchmarks and our OneTracker outperforms other models and achieves state-of-the-art performance.

OneTracker: Unifying Visual Object Tracking with Foundation Models and Efficient Tuning

TL;DR

This work presents a general framework to unify various tracking tasks, termed as One Tracker, which is consisted of Foundation Tracker and Prompt Tracker, and conducts extensive experiments on 6 popular tracking tasks across 11 benchmarks and the One- Tracker outperforms other models and achieves state-of-the-art performance.

Abstract

Visual object tracking aims to localize the target object of each frame based on its initial appearance in the first frame. Depending on the input modility, tracking tasks can be divided into RGB tracking and RGB+X (e.g. RGB+N, and RGB+D) tracking. Despite the different input modalities, the core aspect of tracking is the temporal matching. Based on this common ground, we present a general framework to unify various tracking tasks, termed as OneTracker. OneTracker first performs a large-scale pre-training on a RGB tracker called Foundation Tracker. This pretraining phase equips the Foundation Tracker with a stable ability to estimate the location of the target object. Then we regard other modality information as prompt and build Prompt Tracker upon Foundation Tracker. Through freezing the Foundation Tracker and only adjusting some additional trainable parameters, Prompt Tracker inhibits the strong localization ability from Foundation Tracker and achieves parameter-efficient finetuning on downstream RGB+X tracking tasks. To evaluate the effectiveness of our general framework OneTracker, which is consisted of Foundation Tracker and Prompt Tracker, we conduct extensive experiments on 6 popular tracking tasks across 11 benchmarks and our OneTracker outperforms other models and achieves state-of-the-art performance.
Paper Structure (16 sections, 9 equations, 3 figures, 7 tables)

This paper contains 16 sections, 9 equations, 3 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methodology
  4. Tracking Unification
  5. Foundation Tracker
  6. Prompt Tracker
  7. Training and Inference
  8. Experiments
  9. Implementation Details
  10. Benchmark Results on 6 Tracking Tasks
  11. Ablation Study.
  12. Conclusion
  13. Discussion
  14. Experiment Details
  15. Foundation Tracker Training
  16. ...and 1 more sections

Figures (3)

  • Figure 1: (a) The definition of RGB tracking. (b) The definition of RGB+X tracking. (c) Overview of Foundation Tracker training. (d) The parameter-efficient finetuning of Prompt Tracker.
  • Figure 2: (a) Unified Prompt Embedding structure. (b) Cross Modality Tracking (CMT) Prompters. (c) Tracking Task Perception (TTP) Transformer layers.
  • Figure 3: Visualization results. The blue, red, and green bounding boxes denote ground truth, Foundation Tracker, and Prompt Tracker. Foundation Attn and Prompt Attn denotes the attention map of Foundation Tracker and Prompt Tracker.