Fast Global Localization on Neural Radiance Field

TL;DR

This work tackles the computational bottleneck of global localization on NeRF maps by introducing Fast Loc-NeRF, which combines Monte Carlo Localization with a NeRF map through two key innovations: particle rejection weighting to suppress abnormal rays and a coarse-to-fine multi-scale matching strategy that preserves near-constant work (maintaining $N \cdot B$ approximately constant) while refining pose estimates. The method advances localization efficiency and accuracy by progressively increasing rendering resolution and pixel probes across two refinement stages triggered by pose-variance thresholds $\delta_{\text{refine}}^{1,2}$. Empirical results on LLFF and Zip-NeRF-based setups demonstrate state-of-the-art performance, with faster update steps and improved robustness, especially in challenging, full-room environments. The approach holds practical potential for real-time NeRF-based localization in robotics and AR/VR applications, reducing computational demands without compromising precision.

Abstract

Neural Radiance Fields (NeRF) presented a novel way to represent scenes, allowing for high-quality 3D reconstruction from 2D images. Following its remarkable achievements, global localization within NeRF maps is an essential task for enabling a wide range of applications. Recently, Loc-NeRF demonstrated a localization approach that combines traditional Monte Carlo Localization with NeRF, showing promising results for using NeRF as an environment map. However, despite its advancements, Loc-NeRF encounters the challenge of a time-intensive ray rendering process, which can be a significant limitation in practical applications. To address this issue, we introduce Fast Loc-NeRF, which leverages a coarse-to-fine approach to enable more efficient and accurate NeRF map-based global localization. Specifically, Fast Loc-NeRF matches rendered pixels and observed images on a multi-resolution from low to high resolution. As a result, it speeds up the costly particle update process while maintaining precise localization results. Additionally, to reject the abnormal particles, we propose particle rejection weighting, which estimates the uncertainty of particles by exploiting NeRF's characteristics and considers them in the particle weighting process. Our Fast Loc-NeRF sets new state-of-the-art localization performances on several benchmarks, convincing its accuracy and efficiency.

Figures (6)

• Figure 1: Our Fast Loc-NeRF utilzes particle uncertainty to estimates camera poses with observed image from NeRF map. (a) Initially distributed particles for global localization during the MCL process. (b) Rendered image and uncertainty map from an abnormal particle located outside the valid region. (c) Rendered image and uncertainty map from a normal particle.
• Figure 2: Particle Rejection Weighting We illustrate the particle rejection weighting process. (a) Rendered image from the red and blue particle each, which is located inside and outside room (b) Density distribution $\sigma(z)$ over ray $r(z)=o+zd$ (c) Ray weight distribution $h(z)=T(z)\sigma(z)$ and red region indicate the range between $z_{\text{trans}}$ and $z_{\text{opaque}}$
• Figure 3: Overview of our Coarse-to-Fine Multi-Scale Matching. Our approach shifts focus from exploration to exploitation, increasing the resolution and the number of rendering pixels while we progressively decreases the number of particles in MCL.
• Figure 4: Quantitative comparison with Loc-NeRF. The first row presents position accuracy (%) and position error (cm). The second row shows rotation accuracy (%) and error (cm) on four datasets from LLFF, which are the same datasets conducted on Loc-NeRF paper.
• Figure 5: Ablation studies on threshold of particle rejection weighting.