UncertaintyTrack: Exploiting Detection and Localization Uncertainty in Multi-Object Tracking

TL;DR

UncertaintyTrack addresses the underutilization of detection localization uncertainty in 2D multi-object tracking by integrating probabilistic bounding-box distributions into TBD pipelines. It introduces Prob-YOLOX to produce mean and covariance for bounding boxes and offers four extensions—uncertainty-aware Kalman filtering, confidence ellipse filtering, bounding box relaxation, and entropy-based greedy matching—to leverage uncertainty during data association. Evaluations on BDD100K and MOT17 show reductions in ID switches and improvements in mMOTA, IDF1, and HOTA, underscoring the practical value of probabilistic object detection for MOT. The work also analyzes how detector uncertainty relates to tracking performance and outlines directions for extending these ideas to 3D MOT and to joint uncertainty modeling across detection and feature representations.

Abstract

Multi-object tracking (MOT) methods have seen a significant boost in performance recently, due to strong interest from the research community and steadily improving object detection methods. The majority of tracking methods follow the tracking-by-detection (TBD) paradigm, blindly trust the incoming detections with no sense of their associated localization uncertainty. This lack of uncertainty awareness poses a problem in safety-critical tasks such as autonomous driving where passengers could be put at risk due to erroneous detections that have propagated to downstream tasks, including MOT. While there are existing works in probabilistic object detection that predict the localization uncertainty around the boxes, no work in 2D MOT for autonomous driving has studied whether these estimates are meaningful enough to be leveraged effectively in object tracking. We introduce UncertaintyTrack, a collection of extensions that can be applied to multiple TBD trackers to account for localization uncertainty estimates from probabilistic object detectors. Experiments on the Berkeley Deep Drive MOT dataset show that the combination of our method and informative uncertainty estimates reduces the number of ID switches by around 19\% and improves mMOTA by 2-3%. The source code is available at https://github.com/TRAILab/UncertaintyTrack

Figures (3)

• Figure 1: Examples of tracking results on the BDD100K dataset Yu_2020_BDD100. In these images, orange boxes indicate false negatives, red boxes represent false positives, and blue boxes denote identity switches between tracked objects. The sequence of images is in chronological order, arranged from top to bottom. Compared to our baseline (left), UncertaintyTrack (right) effectively reduces the number of ID switches.
• Figure 2: Overview of UncertaintyTrack. Prob-YOLOX (purple) outputs detections and their covariance matrices (green) which are initially filtered by Confidence Ellipse Filtering (blue). Following the base tracker, UncertaintyTrack applies the filter again before the Box Relaxation step. Matched tracks and remaining detections after this step are passed back to the tracker for track management.
• Figure 3: Examples of ByteTrack$^{*}$zhang2022bytetrack failures on the BDD100K validation set. Orange, red and blue represent false negatives, false positives and ID switches, respectively.