Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reproducing and Extending RaDelft 4D Radar with Camera-Assisted Labels

Kejia Hu, Mohammed Alsakabi, John M. Dolan, Ozan K. Tonguz

TL;DR

This work closes the reproducibility gap for RaDelft's 4D radar semantic segmentation by re-implementing the baseline without source code and providing open training/inference assets. It introduces a camera-assisted labeling pipeline that transfers semantic priors from camera images to radar point clouds, followed by clustering-based refinement to produce accurate radar labels. The authors further study robustness to fog by simulating varying visual degradation and evaluating cross-modal supervision under those conditions. Collectively, the paper delivers an open framework for labeled radar data, demonstrates superior labeling accuracy over the original RaDelft results, and highlights when camera or radar information should dominate under adverse weather.

Abstract

Recent advances in 4D radar highlight its potential for robust environment perception under adverse conditions, yet progress in radar semantic segmentation remains constrained by the scarcity of open source datasets and labels. The RaDelft data set, although seminal, provides only LiDAR annotations and no public code to generate radar labels, limiting reproducibility and downstream research. In this work, we reproduce the numerical results of the RaDelft group and demonstrate that a camera-guided radar labeling pipeline can generate accurate labels for radar point clouds without relying on human annotations. By projecting radar point clouds into camera-based semantic segmentation and applying spatial clustering, we create labels that significantly enhance the accuracy of radar labels. These results establish a reproducible framework that allows the research community to train and evaluate the labeled 4D radar data. In addition, we study and quantify how different fog levels affect the radar labeling performance.

Reproducing and Extending RaDelft 4D Radar with Camera-Assisted Labels

TL;DR

This work closes the reproducibility gap for RaDelft's 4D radar semantic segmentation by re-implementing the baseline without source code and providing open training/inference assets. It introduces a camera-assisted labeling pipeline that transfers semantic priors from camera images to radar point clouds, followed by clustering-based refinement to produce accurate radar labels. The authors further study robustness to fog by simulating varying visual degradation and evaluating cross-modal supervision under those conditions. Collectively, the paper delivers an open framework for labeled radar data, demonstrates superior labeling accuracy over the original RaDelft results, and highlights when camera or radar information should dominate under adverse weather.

Abstract

Recent advances in 4D radar highlight its potential for robust environment perception under adverse conditions, yet progress in radar semantic segmentation remains constrained by the scarcity of open source datasets and labels. The RaDelft data set, although seminal, provides only LiDAR annotations and no public code to generate radar labels, limiting reproducibility and downstream research. In this work, we reproduce the numerical results of the RaDelft group and demonstrate that a camera-guided radar labeling pipeline can generate accurate labels for radar point clouds without relying on human annotations. By projecting radar point clouds into camera-based semantic segmentation and applying spatial clustering, we create labels that significantly enhance the accuracy of radar labels. These results establish a reproducible framework that allows the research community to train and evaluate the labeled 4D radar data. In addition, we study and quantify how different fog levels affect the radar labeling performance.

Paper Structure

This paper contains 25 sections, 11 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Reproducing the RaDelft Baseline
  5. RAED $\rightarrow$ RAE encoding
  6. Backbone (two 2D FPN branches + 3D U-Net refinement)
  7. Loss and class imbalance
  8. Training protocol
  9. Semantic Fusion under Degraded Visual Conditions
  10. Foggy image generation
  11. Radar depth-based segmentation
  12. Camera-radar semantic fusion
  13. Camera-Assisted Labeling
  14. Coordinate Alignment and Calibration
  15. 3-D $\rightarrow$ 2-D Projection
  16. ...and 10 more sections

Figures (4)

  • Figure 1: Example of semantic segmentation results. The first row shows the foggy image at fog intensity of $\beta = 0.15$ and its semantic segmentation. The second row shows the corresponding radar depth map and its segmentation result. The third row shows the clear image and the fused radar-camera segmentation. In semantic segmentation, dark grey represents scenario objects, pink represents pedestrians, blue represents vehicles and red represents bicycles. Plot (b) missed the vehicles due to the fog, but correctly identifies the person (bounded in green box highlighted in the figure). Plot (d) detected vehicles but misclassified the person as a cyclist. Plot (f) combines the vehicle detections from (d) with the correct person classification obtained in (b), showing a superior performance to camera-only (b) and radar-only (d).
  • Figure 2: Radar–camera fusion and radar point cloud labeling pipeline.
  • Figure 3: Visualization of generated radar PCs and the corresponding image and LiDAR ground truth, different colors represents different classes. The color bar distinguishes the 4 classes. Overall, the generated radar PCs correctly identify the vehicles, as in green box, and bicycles, as in orange box.
  • Figure 4: Qualitative comparison of radar point cloud labeling. Our method, in figure (d), correctly classified the cars (points within the green box). Whereas the original RaDelft method failed to classify the vehicles.