Real-Time Spatial Reasoning by Mobile Robots for Reconstruction and Navigation in Dynamic LiDAR Scenes

TL;DR

This paper introduces a brain-inspired framework for real-time spatial reasoning with outdoor LiDAR, enabling simultaneous surface reconstruction and navigation in dynamic environments. It combines single-frame boundary reconstruction via GHPR-based mesh generation with LoS-driven normal estimation and a multi-frame LoS distance-field fusion to identify and remove moving objects, yielding accurate static scene models at about 10 Hz. The approach achieves superior reconstruction quality and real-time performance compared to existing methods, demonstrated on synthetic and real outdoor scenes without reliance on training data or GPUs. This enables robust autonomous navigation and 3D scene understanding in unknown, dynamic outdoor environments.

Abstract

Our brain has an inner global positioning system which enables us to sense and navigate 3D spaces in real time. Can mobile robots replicate such a biological feat in a dynamic environment? We introduce the first spatial reasoning framework for real-time surface reconstruction and navigation that is designed for outdoor LiDAR scanning data captured by ground mobile robots and capable of handling moving objects such as pedestrians. Our reconstruction-based approach is well aligned with the critical cellular functions performed by the border vector cells (BVCs) over all layers of the medial entorhinal cortex (MEC) for surface sensing and tracking. To address the challenges arising from blurred boundaries resulting from sparse single-frame LiDAR points and outdated data due to object movements, we integrate real-time single-frame mesh reconstruction, via visibility reasoning, with robot navigation assistance through on-the-fly 3D free space determination. This enables continuous and incremental updates of the scene and free space across multiple frames. Key to our method is the utilization of line-of-sight (LoS) vectors from LiDAR, which enable real-time surface normal estimation, as well as robust and instantaneous per-voxel free space updates. We showcase two practical applications: real-time 3D scene reconstruction and autonomous outdoor robot navigation in real-world conditions. Comprehensive experiments on both synthetic and real scenes highlight our method's superiority in speed and quality over existing real-time LiDAR processing approaches.

Figures (18)

• Figure 1: Real-time robot navigation and reconstruction of large-scale outdoor scenes from LiDAR point clouds, with moving objects such as pedestrians, presents two significant challenges: (b) First challenge: the sparsity and anisotropy (see red directional arrow and blue directional arrow) of LiDAR point cloud data. Due to this, single-frame data cannot completely describe the scene, making it difficult to establish an understanding of unknown objects or obstacles in real time for navigation and reconstruction. (c) Second challenge: proper handling of moving objects. Points captured on moving object at any time are “outdated” quickly for reconstruction, but still need to be tracked as obstacles to be avoided for robot navigation. (d) Our method addresses these challenges by accurately meshing scenes at single-frame and detecting surrounding free spaces in real time. Results showcased here represent the final scene geometry, with moving objects removed. Our system boasts a response speed of 0.1 second, a critical aspect for scene understanding and path navigation by mobile robots, such as the HUSKY, in dynamic environments.
• Figure 2: Schematic diagram of our spatial reasoning over online laser scans and resemblance to cellular functions in the brain. Inspired by the BVC model lever2009boundary of border cells in the Hippocampus, we implement the construction of the line-of-sight (LOS) distance field of real-time LiDAR scan point clouds through graphical based pipeline. (a) The foundational role of border-sensing cells in the brain. (b) Replicating the role of BVC in our approach. A single frame of LiDAR scan is shown in (1), whose surface boundary is formed by meshes surrounding the point clouds, as indicated by blue dashed lines. Accordingly, the free space inside the boundary is shown as the blue area in (2), where there are no moving object points, which is important for robot navigation. We employ a 3D grid to encode the 3D free spaces and rely on line-of-sight (LoS) information from the robot to the LiDAR scans, for both real-time surface reconstruction and robust estimation of free spaces, as shown in (3). These operations bear resemblance to spatial sensing and tracking in the entorhinal-hippocampal circuit of the brain, where grid cells receive path planning information about obstacles and boundaries in the scene from border cells solstad2008representation.
• Figure 3: An illustration of our real-time spatial reasoning system framework with sample inputs and results.
• Figure 4: Mesh reconstruction in single-frame mode via GHPR inversion: (a) 3D points captured at the current frame, with a selected region coloured in red; (b) points after GHPR inversion and their convex hull; (c) mesh reconstructed from the convex hull connectivity and partial model corresponding to red point clouds (showing internal reconstruction results).
• Figure 5: Illustration of our proposed partition technique for speeding up single-frame mesh reconstruction. The bottom shows the result of using the GHPR algorithm directly, which produces a global watertight model. The top shows that our method uses radial pie-shaped partitioning to adapt to the radial transformation of GHPR and to avoid the distortion of free space detection. Note that we discarded meshes connected to the viewpoint for displaying the interior surface. In fact, each pie-shaped partition is also reconstructed as a watertight model.