EPIPTrack: Rethinking Prompt Modeling with Explicit and Implicit Prompts for Multi-Object Tracking

TL;DR

EPIPTrack addresses robust multi-object tracking under occlusion and appearance similarity by integrating explicit motion-based prompts and implicit appearance prompts into a CLIP-based vision-language framework. It eschews large language models, instead using a TI-Net-generated pseudo-token and a discriminative feature augmentor to align visual and textual embeddings, guided by contrastive, triplet, and similarity-distribution losses. The method achieves state-of-the-art results on MOT17, MOT20, and DanceTrack, demonstrating strong long-term identity preservation and adaptability across scenes, with a plug-and-play design that can augment existing TBD trackers. The work provides practical cross-modal modeling insights and highlights a path toward efficient, end-to-end cross-modal MOT without external language-model dependencies.

Abstract

Multimodal semantic cues, such as textual descriptions, have shown strong potential in enhancing target perception for tracking. However, existing methods rely on static textual descriptions from large language models, which lack adaptability to real-time target state changes and prone to hallucinations. To address these challenges, we propose a unified multimodal vision-language tracking framework, named EPIPTrack, which leverages explicit and implicit prompts for dynamic target modeling and semantic alignment. Specifically, explicit prompts transform spatial motion information into natural language descriptions to provide spatiotemporal guidance. Implicit prompts combine pseudo-words with learnable descriptors to construct individualized knowledge representations capturing appearance attributes. Both prompts undergo dynamic adjustment via the CLIP text encoder to respond to changes in target state. Furthermore, we design a Discriminative Feature Augmentor to enhance visual and cross-modal representations. Extensive experiments on MOT17, MOT20, and DanceTrack demonstrate that EPIPTrack outperforms existing trackers in diverse scenarios, exhibiting robust adaptability and superior performance.

Figures (7)

• Figure 1: Comparison between the proposed framework and mainstream framework.
• Figure 2: A unified vision-language tracking framework, named EPIPTrack. MA and PA refer to motion attributes and body-part attributes, respectively. MN-Corrector is the motion noise corrector.
• Figure 3: Multimodal Interaction Network. We employ a single-layer Multi-Head Cross-Attention (MHCA). [$P_Q$,$P_K$] are learnable positional encodings.
• Figure 4: Robustness evaluation in complex scenarios. Identity interruptions are indicated by color changes. The evaluated target is highlighted for clarity.
• Figure 5: Comparison of attribute fusion variants.