Occlusion-Aware SORT: Observing Occlusion for Robust Multi-Object Tracking

TL;DR

The novel Occlusion-Aware SORT (OA-SORT) framework is presented, a plug-and-play and training-free framework that includes the Occlusion-Aware Module (OAM), the Occlusion-Aware Offset (OAO), and the Bias-Aware Momentum (BAM) that analyzes the occlusion status of objects.

Abstract

Multi-object tracking (MOT) involves analyzing object trajectories and counting the number of objects in video sequences. However, 2D MOT faces challenges due to positional cost confusion arising from partial occlusion. To address this issue, we present the novel Occlusion-Aware SORT (OA-SORT) framework, a plug-and-play and training-free framework that includes the Occlusion-Aware Module (OAM), the Occlusion-Aware Offset (OAO), and the Bias-Aware Momentum (BAM). Specifically, OAM analyzes the occlusion status of objects, where a Gaussian Map (GM) is introduced to reduce background influence. In contrast, OAO and BAM leverage the OAM-described occlusion status to mitigate cost confusion and suppress estimation instability. Comprehensive evaluations on the DanceTrack, SportsMOT, and MOT17 datasets demonstrate the importance of occlusion handling in MOT. On the DanceTrack test set, OA-SORT achieves 63.1% and 64.2% in HOTA and IDF1, respectively. Furthermore, integrating the Occlusion-Aware framework into the four additional trackers improves HOTA and IDF1 by an average of 2.08% and 3.05%, demonstrating the reusability of the occlusion awareness.

Figures (10)

• Figure 1: Performance improvement of OA-SORT over baseline (Hybrid-SORT) on the DanceTrack validation set under different occlusion severity, where the horizontal axis represents the video sequence number. The severity is defined as the proportion of object instances whose average occlusion ratio (Eq. \ref{['eq: Oc_2']}) exceeds 0.75.
• Figure 2: Pipeline of OA-SORT (\ref{['sec: app']}). Before association, OAO (\ref{['sec: OAO']}) leverages the occlusion coefficient, which is derived from KF's estimation by OAM (\ref{['sec: OAM']}) with GM (\ref{['sec: GM']}), to refine the positional cost between trajectories and high-confidence detections. During the update stage, BAM (\ref{['sec: BAM']}) integrates the occlusion coefficient from the latest observation with IoU to refine the KF update.
• Figure 3: Illustration of the depth relationship. Since the bottom edge of the bounding box is parallel to the Baseline, the vertical distance to the Baseline is defined as $\hat{P}^d$.
• Figure 4: The results under GM and different values of $\tau$.
• Figure 5: The example about different $(k_x, k_y)$.