Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RaTrack: Moving Object Detection and Tracking with 4D Radar Point Cloud

Zhijun Pan, Fangqiang Ding, Hantao Zhong, Chris Xiaoxuan Lu

TL;DR

This paper tackles moving object tracking with 4D radar by abandoning bounding-box-centric detection in favor of class-agnostic clustering guided by per-point motion cues. It introduces RaTrack, a fully differentiable, end-to-end framework that combines a backbone for radar feature extraction, a point-wise scene flow estimator, motion-segmentation-based clustering, and a Sinkhorn-based data association to link detections with tracks. The approach yields substantial improvements over LiDAR-oriented baselines on the VoD dataset, highlighting the effectiveness of motion-centric, cluster-based detection for sparse radar data. The work also demonstrates strong ablations showing the importance of the motion estimation module and velocity features, and it provides a solid foundation for radar-based autonomous perception with practical release of code and models.

Abstract

Mobile autonomy relies on the precise perception of dynamic environments. Robustly tracking moving objects in 3D world thus plays a pivotal role for applications like trajectory prediction, obstacle avoidance, and path planning. While most current methods utilize LiDARs or cameras for Multiple Object Tracking (MOT), the capabilities of 4D imaging radars remain largely unexplored. Recognizing the challenges posed by radar noise and point sparsity in 4D radar data, we introduce RaTrack, an innovative solution tailored for radar-based tracking. Bypassing the typical reliance on specific object types and 3D bounding boxes, our method focuses on motion segmentation and clustering, enriched by a motion estimation module. Evaluated on the View-of-Delft dataset, RaTrack showcases superior tracking precision of moving objects, largely surpassing the performance of the state of the art. We release our code and model at https://github.com/LJacksonPan/RaTrack.

RaTrack: Moving Object Detection and Tracking with 4D Radar Point Cloud

TL;DR

This paper tackles moving object tracking with 4D radar by abandoning bounding-box-centric detection in favor of class-agnostic clustering guided by per-point motion cues. It introduces RaTrack, a fully differentiable, end-to-end framework that combines a backbone for radar feature extraction, a point-wise scene flow estimator, motion-segmentation-based clustering, and a Sinkhorn-based data association to link detections with tracks. The approach yields substantial improvements over LiDAR-oriented baselines on the VoD dataset, highlighting the effectiveness of motion-centric, cluster-based detection for sparse radar data. The work also demonstrates strong ablations showing the importance of the motion estimation module and velocity features, and it provides a solid foundation for radar-based autonomous perception with practical release of code and models.

Abstract

Mobile autonomy relies on the precise perception of dynamic environments. Robustly tracking moving objects in 3D world thus plays a pivotal role for applications like trajectory prediction, obstacle avoidance, and path planning. While most current methods utilize LiDARs or cameras for Multiple Object Tracking (MOT), the capabilities of 4D imaging radars remain largely unexplored. Recognizing the challenges posed by radar noise and point sparsity in 4D radar data, we introduce RaTrack, an innovative solution tailored for radar-based tracking. Bypassing the typical reliance on specific object types and 3D bounding boxes, our method focuses on motion segmentation and clustering, enriched by a motion estimation module. Evaluated on the View-of-Delft dataset, RaTrack showcases superior tracking precision of moving objects, largely surpassing the performance of the state of the art. We release our code and model at https://github.com/LJacksonPan/RaTrack.
Paper Structure (17 sections, 5 equations, 4 figures, 2 tables)

This paper contains 17 sections, 5 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Problem Formulation
  4. Proposed Method
  5. Overview
  6. Backbone Network
  7. Motion Estimation Module
  8. Object Detection Module
  9. Data Association Module
  10. End-to-End Training
  11. Experiments
  12. Evaluation Settings
  13. Implementation Details
  14. Overall Performance
  15. Ablation Study
  16. ...and 2 more sections

Figures (4)

  • Figure 1: Overall network pipeline of RaTrack. Note that radar 3D points are shown in the bird's eye view for visualization.
  • Figure 2: Qualitative results of RaTrack. We show on RGB images the ground truth 3D bounding boxes of moving objects (colors indicate different object classes) and projected radar point clouds (colors indicate the distances of points). On corresponding bird's eyes view figures, we show the detected moving objects and their predicted trajectories, where different colors are used to distinguish multiple object clusters.
  • Figure 3: Result comparison between RaTrack and baselines on varying valid object thresholds (both predicted and ground truth objects are filtered) and point-based IoU thresholds.
  • Figure 4: The effect of confidence values on the MOTA and the number of FN and FP. As each method has its own range of recall, we normalised their results from 0.25-1.0 for comparison.