Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CAMELTrack: Context-Aware Multi-cue ExpLoitation for Online Multi-Object Tracking

Vladimir Somers, Baptiste Standaert, Victor Joos, Alexandre Alahi, Christophe De Vleeschouwer

TL;DR

CAMELTrack addresses the limitations of heuristic tracking-by-detection (TbD) by proposing CAMEL, a fully trainable association module that learns context-aware, multi-cue exploitation. CAMEL combines a Temporal Encoder for cue-specific temporal aggregation and a Group-Aware Feature Fusion Encoder to produce disentangled embeddings, enabling single-stage, cue-fused matching with off-the-shelf detectors and predictors. The Association-Centric Training strategy decouples association from detection and cue extraction, using image-free training data and InfoNCE loss to generate robust long-horizon associations. Empirically, CAMELTrack achieves state-of-the-art performance across five MOT benchmarks with efficient training and inference, highlighting the practical value of learned association within the TbD framework.

Abstract

Online multi-object tracking has been recently dominated by tracking-by-detection (TbD) methods, where recent advances rely on increasingly sophisticated heuristics for tracklet representation, feature fusion, and multi-stage matching. The key strength of TbD lies in its modular design, enabling the integration of specialized off-the-shelf models like motion predictors and re-identification. However, the extensive usage of human-crafted rules for temporal associations makes these methods inherently limited in their ability to capture the complex interplay between various tracking cues. In this work, we introduce CAMEL, a novel association module for Context-Aware Multi-Cue ExpLoitation, that learns resilient association strategies directly from data, breaking free from hand-crafted heuristics while maintaining TbD's valuable modularity. At its core, CAMEL employs two transformer-based modules and relies on a novel association-centric training scheme to effectively model the complex interactions between tracked targets and their various association cues. Unlike end-to-end detection-by-tracking approaches, our method remains lightweight and fast to train while being able to leverage external off-the-shelf models. Our proposed online tracking pipeline, CAMELTrack, achieves state-of-the-art performance on multiple tracking benchmarks. Our code is available at https://github.com/TrackingLaboratory/CAMELTrack.

CAMELTrack: Context-Aware Multi-cue ExpLoitation for Online Multi-Object Tracking

TL;DR

CAMELTrack addresses the limitations of heuristic tracking-by-detection (TbD) by proposing CAMEL, a fully trainable association module that learns context-aware, multi-cue exploitation. CAMEL combines a Temporal Encoder for cue-specific temporal aggregation and a Group-Aware Feature Fusion Encoder to produce disentangled embeddings, enabling single-stage, cue-fused matching with off-the-shelf detectors and predictors. The Association-Centric Training strategy decouples association from detection and cue extraction, using image-free training data and InfoNCE loss to generate robust long-horizon associations. Empirically, CAMELTrack achieves state-of-the-art performance across five MOT benchmarks with efficient training and inference, highlighting the practical value of learned association within the TbD framework.

Abstract

Online multi-object tracking has been recently dominated by tracking-by-detection (TbD) methods, where recent advances rely on increasingly sophisticated heuristics for tracklet representation, feature fusion, and multi-stage matching. The key strength of TbD lies in its modular design, enabling the integration of specialized off-the-shelf models like motion predictors and re-identification. However, the extensive usage of human-crafted rules for temporal associations makes these methods inherently limited in their ability to capture the complex interplay between various tracking cues. In this work, we introduce CAMEL, a novel association module for Context-Aware Multi-Cue ExpLoitation, that learns resilient association strategies directly from data, breaking free from hand-crafted heuristics while maintaining TbD's valuable modularity. At its core, CAMEL employs two transformer-based modules and relies on a novel association-centric training scheme to effectively model the complex interactions between tracked targets and their various association cues. Unlike end-to-end detection-by-tracking approaches, our method remains lightweight and fast to train while being able to leverage external off-the-shelf models. Our proposed online tracking pipeline, CAMELTrack, achieves state-of-the-art performance on multiple tracking benchmarks. Our code is available at https://github.com/TrackingLaboratory/CAMELTrack.
Paper Structure (11 sections, 4 equations, 2 figures, 5 tables)

This paper contains 11 sections, 4 equations, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. CAMELTrack Pipeline
  5. Our CAMEL Architecture
  6. Association-Centric Training
  7. Experiments
  8. Datasets and Metrics
  9. Implementation details
  10. Comparison to State-of-the-Art
  11. Ablation Studies

Figures (2)

  • Figure 1: Our proposed CAMEL association module for online tracking learns to produce disentangled tracklet and detection representations by combining various imperfect tracking cues.
  • Figure 2: Architecture overview of CAMELTrack, our online tracking-by-detection pipeline that operates in three steps: (i) object detection, (ii) cue extraction, (iii) single-stage association using our trainable CAMEL module, and (iv) tracklet life cycle management. CAMEL processes the various imperfect cues through two stages: First, the Temporal Encoder (TE) aggregates each cue into tracklet-level representations. Second, the Group-Aware Feature Fusion Encoder (GAFFE) embeds all detection and tracklet cues into a unified discriminative embedding space. The resulting disentangled tracklets and detections representations are finally paired through bipartite matching.