MutualForce: Mutual-Aware Enhancement for 4D Radar-LiDAR 3D Object Detection

TL;DR

This work tackles robust 3D object detection through fusion of 4D radar and LiDAR data, addressing challenges from modality misalignment and sparse radar measurements. It introduces MutualForce, featuring two key modules: IRB, which uses radar indicative features (velocity and RCS) to bidirectionally guide geometric feature learning for both sensors, and SALC, which transfers LiDAR-derived shape information into radar BEV via multi-level shape heatmaps and a robust contrastive loss. The approach achieves state-of-the-art results on the VoD dataset, with $mAP = 71.76\%$ across the entire area and $86.36\%$ in the driving corridor, and notable car AP improvements of about $+4.17\%$ and $+4.20\%$ in different regions, while maintaining real-time performance. Overall, MutualForce demonstrates that leveraging radar-specific properties alongside LiDAR’s shape cues yields more accurate and robust 3D detections in autonomous driving scenarios, particularly under challenging conditions and for small or occluded objects.

Abstract

Radar and LiDAR have been widely used in autonomous driving as LiDAR provides rich structure information, and radar demonstrates high robustness under adverse weather. Recent studies highlight the effectiveness of fusing radar and LiDAR point clouds. However, challenges remain due to the modality misalignment and information loss during feature extractions. To address these issues, we propose a 4D radar-LiDAR framework to mutually enhance their representations. Initially, the indicative features from radar are utilized to guide both radar and LiDAR geometric feature learning. Subsequently, to mitigate their sparsity gap, the shape information from LiDAR is used to enrich radar BEV features. Extensive experiments on the View-of-Delft (VoD) dataset demonstrate our approach's superiority over existing methods, achieving the highest mAP of 71.76% across the entire area and 86.36\% within the driving corridor. Especially for cars, we improve the AP by 4.17% and 4.20% due to the strong indicative features and symmetric shapes.

Figures (5)

• Figure 1: The overall structure of MutualForce.
• Figure 2: The distribution of radar indicative features for different road users in the VoD dataset.
• Figure 3: The Structure of IRB module.
• Figure 4: The structure of SALC module.
• Figure 5: Visualization of the detection on three methods. Ground truths are marked in green, while predicted boxes are red.