VIRUS-NeRF -- Vision, InfraRed and UltraSonic based Neural Radiance Fields

TL;DR

VIRUS-NeRF tackles the challenge of local mapping for autonomous mobile robots using low-cost depth sensors. It extends Instant-NGP by incorporating depth supervision from USS and IRS and by updating a probabilistic occupancy grid to guide ray marching. In 2D experiments, VIRUS-NeRF achieves coverage comparable to LiDAR with environment-dependent accuracy, and ablations show depth supervision and occupancy-grid updates provide substantial gains. The work reports a 46% training speedup over Instant-NGP and provides code for reproducibility.

Abstract

Autonomous mobile robots are an increasingly integral part of modern factory and warehouse operations. Obstacle detection, avoidance and path planning are critical safety-relevant tasks, which are often solved using expensive LiDAR sensors and depth cameras. We propose to use cost-effective low-resolution ranging sensors, such as ultrasonic and infrared time-of-flight sensors by developing VIRUS-NeRF - Vision, InfraRed, and UltraSonic based Neural Radiance Fields. Building upon Instant Neural Graphics Primitives with a Multiresolution Hash Encoding (Instant-NGP), VIRUS-NeRF incorporates depth measurements from ultrasonic and infrared sensors and utilizes them to update the occupancy grid used for ray marching. Experimental evaluation in 2D demonstrates that VIRUS-NeRF achieves comparable mapping performance to LiDAR point clouds regarding coverage. Notably, in small environments, its accuracy aligns with that of LiDAR measurements, while in larger ones, it is bounded by the utilized ultrasonic sensors. An in-depth ablation study reveals that adding ultrasonic and infrared sensors is highly effective when dealing with sparse data and low view variation. Further, the proposed occupancy grid of VIRUS-NeRF improves the mapping capabilities and increases the training speed by 46% compared to Instant-NGP. Overall, VIRUS-NeRF presents a promising approach for cost-effective local mapping in mobile robotics, with potential applications in safety and navigation tasks. The code can be found at https://github.com/ethz-asl/virus nerf.

Figures (6)

• Figure 1: Office: global map in white and VIRUS-NeRF predictions in orange. The axes are the reference frames of the LiDAR and of the two cameras. For this visualization in 3D, VIRUS-NeRF is inferred at multiple heights.
• Figure 2: Experimental setup
• Figure 3: Office and Common Area NND: The first column describes the accuracy and the second one the coverage. The rows show the mean NND and the inlier ($NND < 10cm$) percentage. Each metric is calculated for three zones defined by the GT depth. VIRUS-NeRF is evaluated for 10 runs and the error bar indicates the standard deviation.
• Figure 4: Office: robot in red, global map in grey, GT scan in black and VIRUS-NeRF (USS, IRS & camera) in orange.
• Figure 5: Common Area test point 23: robot in red, global map in grey, GT scan in black and VIRUS-NeRF in orange.