RadSimReal: Bridging the Gap Between Synthetic and Real Data in Radar Object Detection With Simulation

TL;DR

RadSimReal tackles the data bottleneck in radar object detection by offering a hardware-agnostic physical radar simulator that can generate annotated radar images across sensor types and environmental conditions without real-data collection. It treats radar outputs as a 3D convolution of scene reflection points with the measured radar PSF, enabling fast data generation via PSF truncation and sparse convolution. Fidelity analyses show synthetic radar images closely resemble real measurements, with quantitative metrics like the Frechet Inception Distance aligning with real data, and object detectors trained on RadSimReal achieving performance comparable to real-data training, often excelling in cross-dataset scenarios. The approach removes dependence on disclosed radar internals, accelerates data generation, and broadens the applicability of radar-based computer vision for autonomous driving.

Abstract

Object detection in radar imagery with neural networks shows great potential for improving autonomous driving. However, obtaining annotated datasets from real radar images, crucial for training these networks, is challenging, especially in scenarios with long-range detection and adverse weather and lighting conditions where radar performance excels. To address this challenge, we present RadSimReal, an innovative physical radar simulation capable of generating synthetic radar images with accompanying annotations for various radar types and environmental conditions, all without the need for real data collection. Remarkably, our findings demonstrate that training object detection models on RadSimReal data and subsequently evaluating them on real-world data produce performance levels comparable to models trained and tested on real data from the same dataset, and even achieves better performance when testing across different real datasets. RadSimReal offers advantages over other physical radar simulations that it does not necessitate knowledge of the radar design details, which are often not disclosed by radar suppliers, and has faster run-time. This innovative tool has the potential to advance the development of computer vision algorithms for radar-based autonomous driving applications.

Figures (9)

• Figure 1: Comparison between synthetic and real radar images from four different scenarios. Each scenario shows the camera image and the corresponding radar image. (a) and (b) simulation scenarios. (c) and (d) real scenarios.
• Figure 2: Block diagram illustrating the processing steps for conventional simulation (a)+(b) and RadSimReal (a)+(c). (a) Simulates the environment to generate reflection points with RF reflectivity of an automotive scene, while (b) and (c) represent the conventional approach and RadSimReal's approach, respectively, for transforming the reflection points into a radar image.
• Figure 3: Radar image generated with conventional simulation vs. RadSimReal. (a) Radar's PSF 2D slice in range and angle dimensions. (b) Radar image without noise generated by conventional simulation for a scenario with 3 reflection points. (c) Radar image of (b) with noise. (d) Truncated PSF with $99\%$ of its energy. (e) Radar image obtained for the same scenario as in (b) by RadSimReal without noise, (f) The radar image of (e) with noise.
• Figure 4: Comparison between RadSimReal image and a real radar images for the same scenario. (a) Camera image of the scenario. (b) High-resolution LIDAR points segmented by object type. (c) Real radar image. (d) RadSimReal image. The black points in (c) and (d) represent the LIDAR points.
• Figure 5: Qualitative comparison of object detection DNN trained on RadSimReal vs. real data from the RADDet dataset. Rows correspond to different scenarios from RADDet test set. (a) Input radar image, (b) 'U-Net' model's detection score and bounding boxes trained with RADDet. (c) 'U-Net' trained with RadSimReal data. Detected and ground truth bounding boxes marked in pink and white, respectively.