A Probabilistic Formulation of LiDAR Mapping with Neural Radiance Fields

TL;DR

The process through which a Neural Radiance Field (NeRF) can be trained to produce novel LiDAR views of a scene is reexamine and it is shown that by formulating loss as an integral of probability the network can learn multiple peaks for a given ray, allowing the sampling of first, second, or strongest returns from a single output channel.

Abstract

In this paper we reexamine the process through which a Neural Radiance Field (NeRF) can be trained to produce novel LiDAR views of a scene. Unlike image applications where camera pixels integrate light over time, LiDAR pulses arrive at specific times. As such, multiple LiDAR returns are possible for any given detector and the classification of these returns is inherently probabilistic. Applying a traditional NeRF training routine can result in the network learning phantom surfaces in free space between conflicting range measurements, similar to how floater aberrations may be produced by an image model. We show that by formulating loss as an integral of probability (rather than as an integral of optical density) the network can learn multiple peaks for a given ray, allowing the sampling of first, nth, or strongest returns from a single output channel. Code is available at https://github.com/mcdermatt/PLINK

Figures (7)

• Figure 1: The stochastic nature of LiDAR returns means that applying a traditional L2 loss on raw distance values will produce phantom surfaces anywhere there is a conflict between two or more measurements. Rather than requiring a single depth estimate to reproduce multiple measurements at different depths, a better approach is to learn an underlying probability density that reproduces the observed measurements.
• Figure 2: Visualization of the differential probability $\sigma$ that a LiDAR return is generated at a path distance $s$ along a ray that passes through a fence, a semi-transparent glass panel, a shrub, and a brick wall.
• Figure 3: When multiple returns occur on a raypath, the inverse cumulative distribution can be set to a low value to render nearest returns (left) or a high value to render more distant returns (right). The two renderings are different for directions viewing semi-transparent surfaces (windows) but identical for solid surfaces (walls).
• Figure 4: Outline of a forward training pass through the two networks. Uniformly spaced points $\mathbf{\gamma}_i$ (histogram bin centers) are passed to the coarse network. The coarse network attempts to estimate reflectance probability within each bin. Normalizing this result produces a histogram (blue) that guides an importance sampling process. Random draws from the histogram generate a set of test points at which the fine network is evaluated (gray). Loss function (\ref{['eq:coarseToFine']}) steers the coarse-network histogram to match the fine-network integral within each bin.
• Figure 5: A mechanical LiDAR generates a spherical projection, in contrast with cameras, which generate planar projections. Projection errors can be limited by approximating the spherical projection with many planes.