UTrack: Multi-Object Tracking with Uncertain Detections

TL;DR

This work introduces a fast way to obtain the empirical predictive distribution during object detection and incorporate that knowledge in multi-object tracking, and can be integrated into state-of-the-art trackers, enabling them to fully exploit the uncertainty in the detections.

Abstract

The tracking-by-detection paradigm is the mainstream in multi-object tracking, associating tracks to the predictions of an object detector. Although exhibiting uncertainty through a confidence score, these predictions do not capture the entire variability of the inference process. For safety and security critical applications like autonomous driving, surveillance, etc., knowing this predictive uncertainty is essential though. Therefore, we introduce, for the first time, a fast way to obtain the empirical predictive distribution during object detection and incorporate that knowledge in multi-object tracking. Our mechanism can easily be integrated into state-of-the-art trackers, enabling them to fully exploit the uncertainty in the detections. Additionally, novel association methods are introduced that leverage the proposed mechanism. We demonstrate the effectiveness of our contribution on a variety of benchmarks, such as MOT17, MOT20, DanceTrack, and KITTI.

Figures (5)

• Figure 1: Aleatoric uncertainty in bounding-box regression and effect on IoU. (a) Object detection distribution before NMS. After NMS, only point estimates, e.g.$(x_1, y_1, w_1, h_1)\,+$ confidence score, are provided. We measure $\sigma_a$ for $a\in\lbrace x,y,w,h\rbrace$. (b) Illustration of variance propagation to IoU. When the uncertainty is considered, the intersection area, $A_{\cap}$, of two bounding boxes becomes fuzzy (green region), as well as the union of areas.
• Figure 2: Disambiguation of IoU using phase relationships (sequence from MOT17). By knowing that pedestrian 2 carries oscillations and pedestrian 1 does not, the object identities may become more robust to occlusion during tracking.
• Figure 3: Effect, across different datasets, of modifying ByteTrack (baseline) with our proposed methods for exploiting object detector uncertainty. BoT-SORT adds CMC.
• Figure 4: Using homography, as opposed to affine, transformation in the CMC on KITTI.
• Figure 5: Effect, within MOT17, of considering all unmatched detections (from each detection bin) when confirming new tracks.

Theorems & Definitions (1)

• definition thmcounterdefinition