Diffusion-Based Point Cloud Super-Resolution for mmWave Radar Data

TL;DR

The paper tackles the sparsity and ghost artifacts of mmWave radar point clouds in adverse weather by introducing Radar-diffusion, a diffusion-based framework that models the forward degradation from LiDAR BEV to radar BEV via a mean-reverting SDE $d x = \theta_t(\mu - x) dt + \sigma_t d w$ and learns a reverse denoising process using a score network. A novel dual-component objective addresses the imbalanced BEV distribution by separately optimizing target (detections) and blank regions, guided by masks and a weighting parameter $w$. Data are converted to BEV images, with multi-frame fusion (5 frames) and shared FOV alignment, enabling LiDAR supervision to generate dense LiDAR-like BEV outputs from sparse radar BEV inputs. Evaluations on the VOD and RadarHD datasets demonstrate state-of-the-art 3D radar super-resolution and improved downstream registration, highlighting the method’s potential for robust, all-weather perception.

Abstract

The millimeter-wave radar sensor maintains stable performance under adverse environmental conditions, making it a promising solution for all-weather perception tasks, such as outdoor mobile robotics. However, the radar point clouds are relatively sparse and contain massive ghost points, which greatly limits the development of mmWave radar technology. In this paper, we propose a novel point cloud super-resolution approach for 3D mmWave radar data, named Radar-diffusion. Our approach employs the diffusion model defined by mean-reverting stochastic differential equations(SDE). Using our proposed new objective function with supervision from corresponding LiDAR point clouds, our approach efficiently handles radar ghost points and enhances the sparse mmWave radar point clouds to dense LiDAR-like point clouds. We evaluate our approach on two different datasets, and the experimental results show that our method outperforms the state-of-the-art baseline methods in 3D radar super-resolution tasks. Furthermore, we demonstrate that our enhanced radar point cloud is capable of downstream radar point-based registration tasks.

Figures (5)

• Figure 1: Enhancement effect of our method on radar point clouds. The image, raw radar points, enhanced point clouds, and LiDAR point clouds of the corresponding scene are shown in the figure.
• Figure 2: Training process and generating process of our proposed Radar-diffusion. The training process models the degradation of LiDAR BEV image to radar BEV image as the forward diffusion process defined by mean-reverting SDE. By learning the reverse denoising process, the LiDAR-like BEV image is then recovered.
• Figure 3: The data process of LiDAR and radar point clouds.
• Figure 4: Qualitative results of our method and the RadarHD on the VOD dataset. Specifically, the point clouds enhanced by RadarHD are in 2D, while the point clouds enhanced by our method are in 3D.
• Figure 5: Qualitative results of registration on radar point clouds, our enhanced radar point clouds, and LiDAR point cloud using RDMNet shi2023tits. Different colors represent different frames of point clouds.