Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MoRAL: Motion-aware Multi-Frame 4D Radar and LiDAR Fusion for Robust 3D Object Detection

Xiangyuan Peng, Yu Wang, Miao Tang, Bierzynski Kay, Lorenzo Servadei, Robert Wille

TL;DR

MoRAL addresses inter-frame misalignment in multi-frame 4D radar and LiDAR fusion by introducing a Motion-aware Radar Encoder (MRE) that segments moving points and compensates their motion, and a Motion Attention Gated Fusion (MAGF) that injects radar motion cues into LiDAR features to emphasize dynamic foreground objects. The approach leverages MOS-based motion estimation, velocity-augmented radar features, and adaptive feature fusion to produce motion-aware representations, improving detection accuracy for pedestrians and cyclists while maintaining real-time speed. Evaluations on the VoD dataset demonstrate state-of-the-art mAP and AP scores, validating the method’s effectiveness in all-weather fusion and its potential for safer autonomous driving. The work highlights the importance of explicitly modeling object motion in multi-frame radar-LiDAR fusion and provides a path toward more reliable perception in dynamic urban scenes.

Abstract

Reliable autonomous driving systems require accurate detection of traffic participants. To this end, multi-modal fusion has emerged as an effective strategy. In particular, 4D radar and LiDAR fusion methods based on multi-frame radar point clouds have demonstrated the effectiveness in bridging the point density gap. However, they often neglect radar point clouds' inter-frame misalignment caused by object movement during accumulation and do not fully exploit the object dynamic information from 4D radar. In this paper, we propose MoRAL, a motion-aware multi-frame 4D radar and LiDAR fusion framework for robust 3D object detection. First, a Motion-aware Radar Encoder (MRE) is designed to compensate for inter-frame radar misalignment from moving objects. Later, a Motion Attention Gated Fusion (MAGF) module integrate radar motion features to guide LiDAR features to focus on dynamic foreground objects. Extensive evaluations on the View-of-Delft (VoD) dataset demonstrate that MoRAL outperforms existing methods, achieving the highest mAP of 73.30% in the entire area and 88.68% in the driving corridor. Notably, our method also achieves the best AP of 69.67% for pedestrians in the entire area and 96.25% for cyclists in the driving corridor.

MoRAL: Motion-aware Multi-Frame 4D Radar and LiDAR Fusion for Robust 3D Object Detection

TL;DR

MoRAL addresses inter-frame misalignment in multi-frame 4D radar and LiDAR fusion by introducing a Motion-aware Radar Encoder (MRE) that segments moving points and compensates their motion, and a Motion Attention Gated Fusion (MAGF) that injects radar motion cues into LiDAR features to emphasize dynamic foreground objects. The approach leverages MOS-based motion estimation, velocity-augmented radar features, and adaptive feature fusion to produce motion-aware representations, improving detection accuracy for pedestrians and cyclists while maintaining real-time speed. Evaluations on the VoD dataset demonstrate state-of-the-art mAP and AP scores, validating the method’s effectiveness in all-weather fusion and its potential for safer autonomous driving. The work highlights the importance of explicitly modeling object motion in multi-frame radar-LiDAR fusion and provides a path toward more reliable perception in dynamic urban scenes.

Abstract

Reliable autonomous driving systems require accurate detection of traffic participants. To this end, multi-modal fusion has emerged as an effective strategy. In particular, 4D radar and LiDAR fusion methods based on multi-frame radar point clouds have demonstrated the effectiveness in bridging the point density gap. However, they often neglect radar point clouds' inter-frame misalignment caused by object movement during accumulation and do not fully exploit the object dynamic information from 4D radar. In this paper, we propose MoRAL, a motion-aware multi-frame 4D radar and LiDAR fusion framework for robust 3D object detection. First, a Motion-aware Radar Encoder (MRE) is designed to compensate for inter-frame radar misalignment from moving objects. Later, a Motion Attention Gated Fusion (MAGF) module integrate radar motion features to guide LiDAR features to focus on dynamic foreground objects. Extensive evaluations on the View-of-Delft (VoD) dataset demonstrate that MoRAL outperforms existing methods, achieving the highest mAP of 73.30% in the entire area and 88.68% in the driving corridor. Notably, our method also achieves the best AP of 69.67% for pedestrians in the entire area and 96.25% for cyclists in the driving corridor.
Paper Structure (15 sections, 7 equations, 5 figures, 6 tables)

This paper contains 15 sections, 7 equations, 5 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Single-modal 3D Object Detection
  4. Multi-modal 3D Object Detection
  5. Moving Object Segmentation
  6. Proposed Method
  7. Overall Structure
  8. Motion-aware Radar Encoder
  9. Motion Attention Gated Fusion
  10. Experiments
  11. Dataset and Metrics
  12. Main Results
  13. Ablation Study
  14. Discussions
  15. Conclusion

Figures (5)

  • Figure 1: Visualization of the "tail" issue. Ground truth boxes are blue, and predictions are red. (a) shows a single-frame 4D radar point cloud from VoD dataset palffy2022multi. (b) are accumulated multi-frame 4D radar points without motion-aware compensation. (c) denotes accumulated 4D radar point clouds with our MoRAL.
  • Figure 2: The overall structure of our MoRAL.
  • Figure 3: Illustration of segmentation results. (a) shows the velocity threshold-based segmentation, while (b) presents the MOS ground truth. Red and green denote moving and static points. The color intensity of static points reflects the RCS value. Black boxes are ground truth bounding boxes.
  • Figure 4: Architecture of MRE module.
  • Figure 5: Qualitative results comparing our method with MutualForce mutualforce and RLNet xu2024rlnet. Green, yellow, and black boxes denote pedestrians, cyclists, and cars, respectively. Orange and red circles show the false negatives and positives.