Simultaneous Diffusion Sampling for Conditional LiDAR Generation

TL;DR

This paper proposes a novel simultaneous diffusion sampling methodology to generate point clouds conditioned on the 3D structure of the scene as seen from multiple views, allowing it to outperform existing methods by a large margin in a variety of benchmarks.

Abstract

By enabling capturing of 3D point clouds that reflect the geometry of the immediate environment, LiDAR has emerged as a primary sensor for autonomous systems. If a LiDAR scan is too sparse, occluded by obstacles, or too small in range, enhancing the point cloud scan by while respecting the geometry of the scene is useful for downstream tasks. Motivated by the explosive growth of interest in generative methods in vision, conditional LiDAR generation is starting to take off. This paper proposes a novel simultaneous diffusion sampling methodology to generate point clouds conditioned on the 3D structure of the scene as seen from multiple views. The key idea is to impose multi-view geometric constraints on the generation process, exploiting mutual information for enhanced results. Our method begins by recasting the input scan to multiple new viewpoints around the scan, thus creating multiple synthetic LiDAR scans. Then, the synthetic and input LiDAR scans simultaneously undergo conditional generation according to our methodology. Results show that our method can produce accurate and geometrically consistent enhancements to point cloud scans, allowing it to outperform existing methods by a large margin in a variety of benchmarks.

Figures (8)

• Figure 1: Qualitative results of applying our proposed method to LiDAR scene completion. Top row: the single scan input. Bottom row: Many overlapping synthetic scans, resulting in 50% more coverage while retaining a high 80% accuracy. Cyclist and Car coloured in (a,d) for easier readability.
• Figure 1: Qualitative results for densification with and without simultaneous sampling (R2DM Base). Top to bottom: Input, Default Sampling, Simultaneous Sampling, Ground Truth. Our method has stronger geometric consistency, shown by cars and similar objects not merging with walls, and having less blurred sides.
• Figure 2: Overview of our simultaneous diffusion. The input scan is recasted to generate (partial) synthetic scans. We propose a methodology to apply conditional diffusion sampling to all scans simultaneously, achieving more geometrically consistent results.
• Figure 2: More Qualitative results for densification with and without simultaneous sampling (R2DM Base). Top to bottom: Input, Default Sampling, Simultaneous Sampling, Ground Truth. Our simultaneous sampling shows clearer objects in the scene, with less noise and clearer edges between neighbouring objects
• Figure 3: The input LiDAR scan $\mathcal{X}$ is first projected to the equirectangular image $\mathbf{x}$. Then, successive sampling conditioned on image $\mathbf{x}$ is performed to generate the image $\hat{\mathbf{x}}_0$. Finally, this image is backprojected to obtain the generated LiDAR scan $\hat{\mathcal{X}}$.