Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Generative Adversarial Network-based Method for LiDAR-Assisted Radar Image Enhancement

Thakshila Thilakanayake, Oscar De Silva, Thumeera R. Wanasinghe, George K. Mann, Awantha Jayasiri

TL;DR

This work tackles the problem of low-resolution radar imagery in autonomous vehicles under adverse weather by training a Pix2Pix-based GAN to map low-resolution radar inputs to high-resolution LiDAR-ground-truth projections. Ground truth is generated by accumulating LiDAR scans into a global map, cropping relevant instances, and projecting them to 2D ground-truth images aligned with radar data. The method is evaluated on the Boreas dataset using three weather conditions, showing improved image fidelity and object representation in sunny and snowy conditions, with rainy conditions revealing the need for broader training data and augmentation. The approach enables radar-only perception when cameras and LiDAR are unavailable or degraded, offering a promising direction for robust AV sensing in challenging environments.

Abstract

This paper presents a generative adversarial network (GAN) based approach for radar image enhancement. Although radar sensors remain robust for operations under adverse weather conditions, their application in autonomous vehicles (AVs) is commonly limited by the low-resolution data they produce. The primary goal of this study is to enhance the radar images to better depict the details and features of the environment, thereby facilitating more accurate object identification in AVs. The proposed method utilizes high-resolution, two-dimensional (2D) projected light detection and ranging (LiDAR) point clouds as ground truth images and low-resolution radar images as inputs to train the GAN. The ground truth images were obtained through two main steps. First, a LiDAR point cloud map was generated by accumulating raw LiDAR scans. Then, a customized LiDAR point cloud cropping and projection method was employed to obtain 2D projected LiDAR point clouds. The inference process of the proposed method relies solely on radar images to generate an enhanced version of them. The effectiveness of the proposed method is demonstrated through both qualitative and quantitative results. These results show that the proposed method can generate enhanced images with clearer object representation compared to the input radar images, even under adverse weather conditions.

A Generative Adversarial Network-based Method for LiDAR-Assisted Radar Image Enhancement

TL;DR

This work tackles the problem of low-resolution radar imagery in autonomous vehicles under adverse weather by training a Pix2Pix-based GAN to map low-resolution radar inputs to high-resolution LiDAR-ground-truth projections. Ground truth is generated by accumulating LiDAR scans into a global map, cropping relevant instances, and projecting them to 2D ground-truth images aligned with radar data. The method is evaluated on the Boreas dataset using three weather conditions, showing improved image fidelity and object representation in sunny and snowy conditions, with rainy conditions revealing the need for broader training data and augmentation. The approach enables radar-only perception when cameras and LiDAR are unavailable or degraded, offering a promising direction for robust AV sensing in challenging environments.

Abstract

This paper presents a generative adversarial network (GAN) based approach for radar image enhancement. Although radar sensors remain robust for operations under adverse weather conditions, their application in autonomous vehicles (AVs) is commonly limited by the low-resolution data they produce. The primary goal of this study is to enhance the radar images to better depict the details and features of the environment, thereby facilitating more accurate object identification in AVs. The proposed method utilizes high-resolution, two-dimensional (2D) projected light detection and ranging (LiDAR) point clouds as ground truth images and low-resolution radar images as inputs to train the GAN. The ground truth images were obtained through two main steps. First, a LiDAR point cloud map was generated by accumulating raw LiDAR scans. Then, a customized LiDAR point cloud cropping and projection method was employed to obtain 2D projected LiDAR point clouds. The inference process of the proposed method relies solely on radar images to generate an enhanced version of them. The effectiveness of the proposed method is demonstrated through both qualitative and quantitative results. These results show that the proposed method can generate enhanced images with clearer object representation compared to the input radar images, even under adverse weather conditions.
Paper Structure (9 sections, 7 figures, 2 tables)

This paper contains 9 sections, 7 figures, 2 tables.

Table of Contents

  1. INTRODUCTION
  2. Methodology
  3. Dataset
  4. LiDAR Point Cloud Map Generation
  5. LiDAR Point Cloud Map Cropping and 2D Projection
  6. GAN Model and Training
  7. Evaluation Metrics
  8. RESULTS
  9. CONCLUSIONS

Figures (7)

  • Figure 1: Illustration of LiDAR, camera, and radar images of a sample location from the Boreas dataset for sunny and snowy weather conditions: (a) 2D projected, raw LIDAR scan for sunny weather conditions, (b) 2D projected, raw LIDAR scan for snowy weather conditions, (c) camera image for sunny weather conditions, (d) camera image for snowy weather conditions, (e) radar image for sunny weather conditions, (f) radar image for snowy weather conditions.
  • Figure 2: Illustration of the overall system diagram of the proposed radar image enhancement method
  • Figure 3: Illustration of the inference process of the proposed radar image enhancement method
  • Figure 4: Illustration of pixel-wise matching (a) radar image, (b) 2D projected, raw LIDAR scan, (c) 2D projected cropped instance of a LiDAR point cloud map of a sample location from the Boreas dataset.
  • Figure 5: Illustration of sample results of dataset 1, which was collected in good weather conditions. Row 1: input radar images. Row 2: enlarged versions of the red rectangular area of the images in row 1. Row 3: enhanced images generated by the GAN. Row 4: enlarged version of the red rectangular area of the images in row 3. Row 5: corresponding ground truth images. Sample numbers are shown at the top of the figure.
  • ...and 2 more figures