SimpleTrack: Understanding and Rethinking 3D Multi-object Tracking

TL;DR

SimpleTrack analyzes 3D MOT as a four-part pipeline (pre-processing, motion model, association, life cycle management), identifies common failure modes, and introduces targeted, simple improvements such as stricter NMS, 3D GIoU, and a two-stage association. The method achieves competitive to state-of-the-art results on Waymo Open Dataset and nuScenes, and the authors additionally critique benchmarking practices (detection frequency and tracklet interpolation) to inform future evaluation. Key contributions include a robust, learning-free baseline that performs strongly across datasets, and insights into how benchmark design shapes real-world tracking performance. The work points toward integrating long-term information and learning-based intra-frame reasoning to further advance 3D MOT beyond purely tracking-by-detection.

Abstract

3D multi-object tracking (MOT) has witnessed numerous novel benchmarks and approaches in recent years, especially those under the "tracking-by-detection" paradigm. Despite their progress and usefulness, an in-depth analysis of their strengths and weaknesses is not yet available. In this paper, we summarize current 3D MOT methods into a unified framework by decomposing them into four constituent parts: pre-processing of detection, association, motion model, and life cycle management. We then ascribe the failure cases of existing algorithms to each component and investigate them in detail. Based on the analyses, we propose corresponding improvements which lead to a strong yet simple baseline: SimpleTrack. Comprehensive experimental results on Waymo Open Dataset and nuScenes demonstrate that our final method could achieve new state-of-the-art results with minor modifications. Furthermore, we take additional steps and rethink whether current benchmarks authentically reflect the ability of algorithms for real-world challenges. We delve into the details of existing benchmarks and find some intriguing facts. Finally, we analyze the distribution and causes of remaining failures in \name\ and propose future directions for 3D MOT. Our code is available at https://github.com/TuSimple/SimpleTrack.

Figures (9)

• Figure 1: 3D MOT pipeline. For simplicity, we only visualize the steps between frame k and frame k+1. Best view in color.
• Figure 2: Comparison between score filtering and NMS. To remove the redundant bounding boxes on row 2, score filtering needs at least a 0.24 threshold, but this will eliminate the detections on row 1. However, NMS can well satisfy both by removing the overlapping on row 2 and maintaining the recall on row 1.
• Figure 3: Illustration of association metrics. Left: IoU versus GIoU. Right: L2 Distance versus GIoU. Details are in Sec. \ref{['subsubsec::asso_metrics']}.
• Figure 4: Comparison of association metrics on WOD (left & middle) and nuScenes (right). "M-Dis" is the short for Mahalanobis distance. The best method is closest to the bottom-right corner, having the lowest ID-Switches and highest MOTA/AMOTA.
• Figure 5: Comparison of matching strategies on WOD.