Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LiDAR Data Synthesis with Denoising Diffusion Probabilistic Models

Kazuto Nakashima, Ryo Kurazume

TL;DR

This work addresses the challenge of generating realistic LiDAR data by introducing R2DM, a denoising diffusion probabilistic model that operates on a range–reflectance image representation. By incorporating explicit spatial bias through Fourier features (and spherical-harmonics variants) and a carefully designed data representation, R2DM achieves state-of-the-art fidelity and efficiency on KITTI-360 and KITTI-Raw, outperforming prior diffusion and GAN-based approaches. The authors further extend the model to a flexible LiDAR completion pipeline using RePaint, enabling robust beam-level upsampling under various corruptions. Overall, R2DM advances high-fidelity LiDAR synthesis and restoration with practical implications for scalable simulation and perception tasks in autonomous systems.

Abstract

Generative modeling of 3D LiDAR data is an emerging task with promising applications for autonomous mobile robots, such as scalable simulation, scene manipulation, and sparse-to-dense completion of LiDAR point clouds. While existing approaches have demonstrated the feasibility of image-based LiDAR data generation using deep generative models, they still struggle with fidelity and training stability. In this work, we present R2DM, a novel generative model for LiDAR data that can generate diverse and high-fidelity 3D scene point clouds based on the image representation of range and reflectance intensity. Our method is built upon denoising diffusion probabilistic models (DDPMs), which have shown impressive results among generative model frameworks in recent years. To effectively train DDPMs in the LiDAR domain, we first conduct an in-depth analysis of data representation, loss functions, and spatial inductive biases. Leveraging our R2DM model, we also introduce a flexible LiDAR completion pipeline based on the powerful capabilities of DDPMs. We demonstrate that our method surpasses existing methods in generating tasks on the KITTI-360 and KITTI-Raw datasets, as well as in the completion task on the KITTI-360 dataset. Our project page can be found at https://kazuto1011.github.io/r2dm.

LiDAR Data Synthesis with Denoising Diffusion Probabilistic Models

TL;DR

This work addresses the challenge of generating realistic LiDAR data by introducing R2DM, a denoising diffusion probabilistic model that operates on a range–reflectance image representation. By incorporating explicit spatial bias through Fourier features (and spherical-harmonics variants) and a carefully designed data representation, R2DM achieves state-of-the-art fidelity and efficiency on KITTI-360 and KITTI-Raw, outperforming prior diffusion and GAN-based approaches. The authors further extend the model to a flexible LiDAR completion pipeline using RePaint, enabling robust beam-level upsampling under various corruptions. Overall, R2DM advances high-fidelity LiDAR synthesis and restoration with practical implications for scalable simulation and perception tasks in autonomous systems.

Abstract

Generative modeling of 3D LiDAR data is an emerging task with promising applications for autonomous mobile robots, such as scalable simulation, scene manipulation, and sparse-to-dense completion of LiDAR point clouds. While existing approaches have demonstrated the feasibility of image-based LiDAR data generation using deep generative models, they still struggle with fidelity and training stability. In this work, we present R2DM, a novel generative model for LiDAR data that can generate diverse and high-fidelity 3D scene point clouds based on the image representation of range and reflectance intensity. Our method is built upon denoising diffusion probabilistic models (DDPMs), which have shown impressive results among generative model frameworks in recent years. To effectively train DDPMs in the LiDAR domain, we first conduct an in-depth analysis of data representation, loss functions, and spatial inductive biases. Leveraging our R2DM model, we also introduce a flexible LiDAR completion pipeline based on the powerful capabilities of DDPMs. We demonstrate that our method surpasses existing methods in generating tasks on the KITTI-360 and KITTI-Raw datasets, as well as in the completion task on the KITTI-360 dataset. Our project page can be found at https://kazuto1011.github.io/r2dm.
Paper Structure (29 sections, 7 equations, 6 figures, 4 tables)

This paper contains 29 sections, 7 equations, 6 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Generative models
  4. LiDAR data generation
  5. Proposed Method
  6. Preliminary
  7. Forward diffusion process
  8. Reverse diffusion process
  9. Training
  10. Sampling
  11. Loss function
  12. Data representation
  13. Spatial inductive bias
  14. Noise prediction model
  15. LiDAR completion
  16. ...and 14 more sections

Figures (6)

  • Figure 1: LiDAR upsampling using our diffusion model. We show the sparse LiDAR inputs and our upsampled results as point clouds (top), range images (middle), and reflectance images (bottom). Our results were obtained through image-based conditional generation using our model. The range and reflectance images are partially zoomed for visual purposes.
  • Figure 2: Overview of R2DM. (a) The diffusion processes are performed on the range/reflectance image representation. (b) U-Net is trained to recursively denoise the latent variables $z_t$ at $t>0$, conditioned by the beam angle-based spatial bias and the scheduled signal-to-noise ratio $\lambda_t$.
  • Figure 3: Conditional generation using R2DM. We showcase the simulated corruptions (top) and our restored results (middle). Our method can handle the various levels of completion. The road removal (right) mimics the wet road situation.
  • Figure 4: Comparison of diffusion-based methods. For overall metrics, our method achieved better scores with the significantly lower number of function evaluations (NFE), against 1160 steps of LiDARGen zyrianov2022learning.
  • Figure 5: Unconditional generation results. We show the results of the baselines and our method on KITTI-360 (top) and KITTI-Raw (bottom). The LiDARGen results are from officially released samples zyrianov2022learning. The DUSty v2 results are generated using the official pre-trained models nakashima2023generative.
  • ...and 1 more figures