RadarNeXt: Real-Time and Reliable 3D Object Detector Based On 4D mmWave Imaging Radar

TL;DR

RadarNeXt addresses the need for real-time, reliable 3D object detection from 4D mmWave radar point clouds on edge devices. It leverages a Re-parameterizable Depthwise Convolution (Rep-DWC) backbone to reduce parameters and accelerate inference, and a Multi-path Deformable Foreground Enhancement Network (MDFEN) as the neck to enhance sparse foreground geometry via DCNv3. An anchor-free CenterHead with a composite loss balances classification and 3D bbox regression using $L_{FL}$, $L_{L1}$, $L_{IoU}$, $L_{dIoU}$, and $L_{mse}$. On VoD and TJ4D datasets, RadarNeXt achieves 50.48 mAP and 32.30 mAP, with inference speeds of 67.10 FPS on the RTX A4000 and 28.40 FPS on the Jetson AGX Orin, demonstrating competitive accuracy with superior edge efficiency. The results validate that deferring sparsity handling to the fusion stage via MDFEN yields a practical balance between accuracy and real-time performance for automotive radar perception.

Abstract

3D object detection is crucial for Autonomous Driving (AD) and Advanced Driver Assistance Systems (ADAS). However, most 3D detectors prioritize detection accuracy, often overlooking network inference speed in practical applications. In this paper, we propose RadarNeXt, a real-time and reliable 3D object detector based on the 4D mmWave radar point clouds. It leverages the re-parameterizable neural networks to catch multi-scale features, reduce memory cost and accelerate the inference. Moreover, to highlight the irregular foreground features of radar point clouds and suppress background clutter, we propose a Multi-path Deformable Foreground Enhancement Network (MDFEN), ensuring detection accuracy while minimizing the sacrifice of speed and excessive number of parameters. Experimental results on View-of-Delft and TJ4DRadSet datasets validate the exceptional performance and efficiency of RadarNeXt, achieving 50.48 and 32.30 mAPs with the variant using our proposed MDFEN. Notably, our RadarNeXt variants achieve inference speeds of over 67.10 FPS on the RTX A4000 GPU and 28.40 FPS on the Jetson AGX Orin. This research demonstrates that RadarNeXt brings a novel and effective paradigm for 3D perception based on 4D mmWave radar.

Figures (5)

• Figure 1: Our RadarNeXt enables reliable and efficient real-time 3D detection on edge devices like the Jetson AGX Orin, which has limited computational and energy resources.
• Figure 2: The overview of the architecture of our RadarNeXt. ①, ②, ③, ④, and ⑤ are potential positions of DCNv3 in the MDFEN variants for the following ablation studies to validate the current design that DCNv3 on the position ④.
• Figure 3: The architecture of Rep-DWC proposed by MobileOne. Pink and green are down-sampling layers with “$stride=2$”. Orange and Blue are normal layers with “$stride=1$”.
• Figure 4: Qualitative results on the VoD dataset: Images are the image of driving scenes. GTs are the ground truths. The red dots within figures are radar points. Red cuboids represent the Car. Blue cuboids represent the Cyclist. Green cuboids represent the Pedestrian. FPN shows the results of RadarNeXt (FPN). PAN shows the results of RadarNeXt (PAN). MDFEN shows the results of RadarNeXt (MDFEN).
• Figure 5: Qualitative results on the TJ4D dataset: Images are the image of driving scenes. GTs are the ground truths. The red dots within figures are radar points. Red cuboids represent the Car. Blue cuboids represent the Cyclist. Green cuboids represent the Pedestrian. Purple cuboids represent the Truck. FPN shows the results of RadarNeXt (FPN). PAN shows the results of RadarNeXt (PAN). MDFEN shows the results of RadarNeXt (MDFEN).