Real-time Neural Dense Elevation Mapping for Urban Terrain with Uncertainty Estimations

TL;DR

This work designs a novel pre-processing and point features representation approach that ensures high robustness and computational efficiency when integrating multiple point cloud frames, and develops a generative Bayesian model that recovers the detailed terrain structures while simultaneously providing the pixel-wise reconstruction uncertainty.

Abstract

Having good knowledge of terrain information is essential for improving the performance of various downstream tasks on complex terrains, especially for the locomotion and navigation of legged robots. We present a novel framework for neural urban terrain reconstruction with uncertainty estimations. It generates dense robot-centric elevation maps online from sparse LiDAR observations. We design a novel pre-processing and point features representation approach that ensures high robustness and computational efficiency when integrating multiple point cloud frames. A Bayesian-GAN model then recovers the detailed terrain structures while simultaneously providing the pixel-wise reconstruction uncertainty. We evaluate the proposed pipeline through extensive simulation and real-world experiments. It demonstrates efficient terrain reconstruction with high quality and real-time performance on a mobile platform, which further benefits the downstream tasks of legged robots. (See https://kin-zhang.github.io/ndem/ for more details.)

Figures (7)

• Figure 1: We present an online neural dense elevation mapping approach that recovers detailed urban terrain structures from sparse and noisy $16$-channel LiDAR observations (a)(b), while additionally providing the terrain reconstruction uncertainty (c). The framework can further benefit the downstream locomotion and navigation tasks, especially for legged robots.
• Figure 2: Examples of the simulation environment. The generated maps contain diverse types of urban terrains. A $16$-channel LiDAR moves and observes the environment. The LiDAR's orientations change with the local topography considering the robot's foot configuration (red balls).
• Figure 3: The proposed elevation mapping pipeline. It first pre-processes the observed sparse point clouds to update the statistical point features. A generative Bayesian model then encodes the current point features to generate the dense elevation map and the pixel-wise reconstruction uncertainty while simultaneously returning a binary edge map for multitask learning in the training phase. Two discriminators are adopted to guide the learning of elevation map and edge map generation using adversarial loss $\mathcal{L}^{adv}$ and feature matching loss $\mathcal{L}^{fm}$ (See Section \ref{['sec:training']} for details of the loss terms).
• Figure 4: Visualized examples of the elevation mapping experiments in a simulated urban scenario (the first row, $\qtyproduct[product-units=single]{8 x 8}{\metre}$), simulated hill scenario (the second row, $\qtyproduct[product-units=single]{8 x 8}{\metre}$), and on real-world stairs (the third row, $\qtyproduct[product-units=single]{3.2 x 3.2}{\metre}$) using different approaches, where different colors indicate the height values. For urban terrains and real-world stairs, our approach (N.D.E.M.) provides accurate dense elevation maps with high reconstruction quality and can further recover detailed terrain structures by introducing uncertainty estimation and adopting our statistical point features. In the hills scenario, our approach can still recover the terrain in a global view. However, all of our models fail to recover the local features and wrongly generate smooth and step-like terrains.
• Figure 5: The percentage (orange bar plot) of the grids with different levels of error and the corresponding estimated $\boldsymbol{\sigma}$ values (purple curve). Our approach provides high-quality mapping results that are accurate enough for various downstream tasks. The uncertainty estimations further reflect the confidence of our model on the generated maps.