Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BiTrack: Bidirectional Offline 3D Multi-Object Tracking Using Camera-LiDAR Data

Kemiao Huang, Yinqi Chen, Meiying Zhang, Qi Hao

TL;DR

BiTrack tackles offline 3D multi-object tracking from camera-LiDAR data by integrating point-level 2D-3D fusion, robust initial trajectory generation, and bidirectional trajectory re-optimization. It introduces a density-based point-level 2D-3D registration, an integrated motion similarity metric with tracklet recovery and velocity re-initialization, and a greedy, fragment-level bidirectional fusion and refinement scheme. The approach yields state-of-the-art results on KITTI for 3D OMOT, with improved DetA and HOTA, while remaining computation-efficient and fully automated. BiTrack's modular design makes it suitable for large-scale 3D annotation and offline tracking tasks in multi-sensor environments.

Abstract

Compared with real-time multi-object tracking (MOT), offline multi-object tracking (OMOT) has the advantages to perform 2D-3D detection fusion, erroneous link correction, and full track optimization but has to deal with the challenges from bounding box misalignment and track evaluation, editing, and refinement. This paper proposes "BiTrack", a 3D OMOT framework that includes modules of 2D-3D detection fusion, initial trajectory generation, and bidirectional trajectory re-optimization to achieve optimal tracking results from camera-LiDAR data. The novelty of this paper includes threefold: (1) development of a point-level object registration technique that employs a density-based similarity metric to achieve accurate fusion of 2D-3D detection results; (2) development of a set of data association and track management skills that utilizes a vertex-based similarity metric as well as false alarm rejection and track recovery mechanisms to generate reliable bidirectional object trajectories; (3) development of a trajectory re-optimization scheme that re-organizes track fragments of different fidelities in a greedy fashion, as well as refines each trajectory with completion and smoothing techniques. The experiment results on the KITTI dataset demonstrate that BiTrack achieves the state-of-the-art performance for 3D OMOT tasks in terms of accuracy and efficiency.

BiTrack: Bidirectional Offline 3D Multi-Object Tracking Using Camera-LiDAR Data

TL;DR

BiTrack tackles offline 3D multi-object tracking from camera-LiDAR data by integrating point-level 2D-3D fusion, robust initial trajectory generation, and bidirectional trajectory re-optimization. It introduces a density-based point-level 2D-3D registration, an integrated motion similarity metric with tracklet recovery and velocity re-initialization, and a greedy, fragment-level bidirectional fusion and refinement scheme. The approach yields state-of-the-art results on KITTI for 3D OMOT, with improved DetA and HOTA, while remaining computation-efficient and fully automated. BiTrack's modular design makes it suitable for large-scale 3D annotation and offline tracking tasks in multi-sensor environments.

Abstract

Compared with real-time multi-object tracking (MOT), offline multi-object tracking (OMOT) has the advantages to perform 2D-3D detection fusion, erroneous link correction, and full track optimization but has to deal with the challenges from bounding box misalignment and track evaluation, editing, and refinement. This paper proposes "BiTrack", a 3D OMOT framework that includes modules of 2D-3D detection fusion, initial trajectory generation, and bidirectional trajectory re-optimization to achieve optimal tracking results from camera-LiDAR data. The novelty of this paper includes threefold: (1) development of a point-level object registration technique that employs a density-based similarity metric to achieve accurate fusion of 2D-3D detection results; (2) development of a set of data association and track management skills that utilizes a vertex-based similarity metric as well as false alarm rejection and track recovery mechanisms to generate reliable bidirectional object trajectories; (3) development of a trajectory re-optimization scheme that re-organizes track fragments of different fidelities in a greedy fashion, as well as refines each trajectory with completion and smoothing techniques. The experiment results on the KITTI dataset demonstrate that BiTrack achieves the state-of-the-art performance for 3D OMOT tasks in terms of accuracy and efficiency.

Paper Structure

This paper contains 21 sections, 10 equations, 5 figures, 7 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. System Setup and Problem Statement
  4. Proposed Methods
  5. 2D-3D Object Registration via Points
  6. Point-Level Segmentation
  7. Point-Level Association
  8. Reliable Initial Trajectory Generation
  9. Integrated Object Motion similarity Metric
  10. Previous Tracklet Recovery and Double Miss Thresholds
  11. Velocity Re-initialization
  12. Efficient Trajectory Re-Optimization
  13. Bidirectional Trajectory Clustering
  14. Bidirectional Trajectory Fusion
  15. Single-Trajectory Refinement
  16. ...and 6 more sections

Figures (5)

  • Figure 1: A post-processing based offline 3D multi-object tracking framework with three main stages.
  • Figure 2: The system architecture of BiTrack.
  • Figure 3: A visual explanation of the symbol definitions for bidirectional trajectory fusion. Best viewed in color.
  • Figure 4: The bird's-eye view of 2D-3D object fusion results on a sequence. The gray boxes are the objects that the dataset doesn't care. The green boxes are true positives. The blue boxes are false positives. The red boxes are false negatives.
  • Figure 5: Visualization of the bidirectional trajectory fusion. Circles represent detected boxes and arrows are directions.