Exploring Enhanced Contextual Information for Video-Level Object Tracking

TL;DR

MCITrack introduces a video-level tracking framework that propagates rich contextual information across frames by leveraging Mamba hidden states within a Contextual Information Fusion (CIF) module. The CIF module, consisting of a Mamba layer and cross-attention mechanisms, stores historical context and integrally injects it into backbone features to enhance multi-level representations during tracking. Extensive experiments on LaSOT, GOT-10k, TrackingNet, and other benchmarks demonstrate state-of-the-art performance, with notable improvements over prior methods and detailed ablations validating the design choices. The approach advances video-level tracking by enabling richer, more robust context transmission, though it incurs higher training cost and computational overhead, suggesting directions for efficiency improvements.

Abstract

Contextual information at the video level has become increasingly crucial for visual object tracking. However, existing methods typically use only a few tokens to convey this information, which can lead to information loss and limit their ability to fully capture the context. To address this issue, we propose a new video-level visual object tracking framework called MCITrack. It leverages Mamba's hidden states to continuously record and transmit extensive contextual information throughout the video stream, resulting in more robust object tracking. The core component of MCITrack is the Contextual Information Fusion module, which consists of the mamba layer and the cross-attention layer. The mamba layer stores historical contextual information, while the cross-attention layer integrates this information into the current visual features of each backbone block. This module enhances the model's ability to capture and utilize contextual information at multiple levels through deep integration with the backbone. Experiments demonstrate that MCITrack achieves competitive performance across numerous benchmarks. For instance, it gets 76.6% AUC on LaSOT and 80.0% AO on GOT-10k, establishing a new state-of-the-art performance. Code and models are available at https://github.com/kangben258/MCITrack.

Figures (5)

• Figure 1: Comparison of different contextual information propagation methods. (a) Using dynamic templates to record the object's shape. (b) Using extra tokens to transmit contextual information. (c) Using hidden state layers to transmit contextual information.
• Figure 2: (a) Framework of the proposed MCITrack. The key components include the backbone for visual feature extraction, the contextual information fusion module for recording and transmitting contextual information, and the prediction head. (b) Architecture of the proposed Contextual Information Fusion(CIF) block.
• Figure 3: The detailed mamba layer in the CIF block.
• Figure 4: EAO rank plots on VOT2020.
• Figure 5: Visualization of the output features of CIF module.