Enhancing Autonomous Navigation by Imaging Hidden Objects using Single-Photon LiDAR

TL;DR

Robots struggle to detect hidden obstacles around corners, limiting safe and efficient navigation. This work presents a three-module NLOS pipeline that uses multi-bounce histograms from a commercial SPAD-based single-photon LiDAR to estimate occupancy maps of occluded regions with a CNN, and then plans safe paths from these estimates. The authors contribute (1) the first experimental demonstration of NLOS imaging for autonomous navigation, (2) a data-driven hidden-object occupancy estimator, and (3) a dynamics-integrated transient rendering framework in Project Chrono for realistic simulations. Real-world experiments show smoother, faster navigation around hidden objects, and simulations indicate improved collision avoidance at higher speeds, highlighting a pathway to safer navigation in complex environments without additional infrastructure.

Abstract

Robust autonomous navigation in environments with limited visibility remains a critical challenge in robotics. We present a novel approach that leverages Non-Line-of-Sight (NLOS) sensing using single-photon LiDAR to improve visibility and enhance autonomous navigation. Our method enables mobile robots to "see around corners" by utilizing multi-bounce light information, effectively expanding their perceptual range without additional infrastructure. We propose a three-module pipeline: (1) Sensing, which captures multi-bounce histograms using SPAD-based LiDAR; (2) Perception, which estimates occupancy maps of hidden regions from these histograms using a convolutional neural network; and (3) Control, which allows a robot to follow safe paths based on the estimated occupancy. We evaluate our approach through simulations and real-world experiments on a mobile robot navigating an L-shaped corridor with hidden obstacles. Our work represents the first experimental demonstration of NLOS imaging for autonomous navigation, paving the way for safer and more efficient robotic systems operating in complex environments. We also contribute a novel dynamics-integrated transient rendering framework for simulating NLOS scenarios, facilitating future research in this domain.

Figures (5)

• Figure 1: Autonomous navigation using single-photon LiDAR. (a) Autonomous navigation is challenging under unseen threats such as a cyclist emerging from a blind corner. We leverage single-photon LiDAR for autonomous navigation by seeing around corners using indirect reflections of light. (b) With only line-of-sight perception, the location of hidden objects around the corner is unknown to the agent, resulting in collisions. Our approach enables non-line-of-sight (NLOS) perception with the estimated occupancy map accurately estimating the hidden object location. (c) We evaluate our NLOS-based navigation approach experimentally using a commercial-grade single-photon LiDAR mounted on a two-wheeled robot. Incorporating NLOS perception results in a smoother trajectory while avoiding the hidden obstacle.
• Figure 2: Our pipeline for NLOS-aided Autonomous Navigation. (a) Sensing: We capture multi-bounce histograms using single-photon LiDAR to gather information about hidden regions. (b) Perception: Captured histograms are then processed using a data-driven approach to estimate the occupancy map of the occluded region. (c) Control: Then the optimal path is planned based on the estimated occupancy map for navigating around obstacles.
• Figure 3: Simulation Results for navigating a L-turn corner using line-of-sight and non-line-of-sight perception. (a) Comparison of collision avoidance success rate between LOS and NLOS perception. NLOS has an edge in performance over LOS at higher speeds where path planning cannot react on time. (b) And, we find that higher localization error leads to worse performance in avoiding obstacles. Which matches intuition.
• Figure 4: Capture and evaluation setup. (a) To validate the perception algorithm and generate a dataset to train the CNN, we utilize a 2-axis gantry to precisely position a hidden object (in this case, a retro-reflective arrow) in some region outside the FoV of the SPAD sensor. For training, we generate data for many positions and orientations of both the sensor and object to produce a diverse dataset. (b) Multi-bounce histograms of four of the output zones from the SPAD sensor. The first bounce is the primary spike, and in later bins (i.e. with a longer time-of-flight), the third bounce reflection from the object can be observed. Different zones have a different view of the scene, and so the relative intensity and peak position encode the object's state. This information is leveraged by the model to predict the position of the hidden object. (c) The estimates of our model on an evaluation set (i.e., data held out from training) which shows our model estimates object position with sub-10 cm accuracy. We found that evaluation performance decreased substantially when testing on states (i.e., camera/object positions) not seen during training.
• Figure 5: Blind corner navigation using non-line-of-sight perception on a physical robot. To test the effectiveness of non-line-of-sight imaging for autonomous navigation, we ran experiments on a physical robot in an L-shaped blind corner. An object was placed in an area hidden from view of the sensor. (a) The agent with only a LOS view couldn't prepare for the object before rounding the corner, and had to overshoot back inside to reach the goal. (b) On the other hand, when NLOS information was used, the robot could successfully generate and follow a path which reduced the wasted travel time. (c) A quantitative depiction of the efficiency improvements for an NLOS-capable robot. With only LOS information, the robot took more than 2$\times$ longer and required a 33% longer trajectory.