NARUTO: Neural Active Reconstruction from Uncertain Target Observations

TL;DR

NARUTO tackles neural active reconstruction in large-scale, unconstrained 3D environments by uniting a hybrid neural mapping backbone with a learnable uncertainty representation and an uncertainty-guided planning module. The system uses a multi-resolution hash-grid as the mapping backbone for fast, high-frequency surface capture, coupled with an explicit uncertainty volume to drive goal search and efficient path planning via uncertainty-aware RRT. Key contributions include a 6DoF active reconstruction framework, an uncertainty aggregation strategy for selecting informative goals, an Active Ray Sampling scheme to stabilize mapping, and demonstrated state-of-the-art performance on Replica and MP3D benchmarks with substantial gains in completeness (from 73% to 90%). The work advances real-time, uncertainty-aware navigation and mapping, enabling more robust autonomous exploration and improved neural SLAM capabilities in complex indoor scenes.

Abstract

We present NARUTO, a neural active reconstruction system that combines a hybrid neural representation with uncertainty learning, enabling high-fidelity surface reconstruction. Our approach leverages a multi-resolution hash-grid as the mapping backbone, chosen for its exceptional convergence speed and capacity to capture high-frequency local features.The centerpiece of our work is the incorporation of an uncertainty learning module that dynamically quantifies reconstruction uncertainty while actively reconstructing the environment. By harnessing learned uncertainty, we propose a novel uncertainty aggregation strategy for goal searching and efficient path planning. Our system autonomously explores by targeting uncertain observations and reconstructs environments with remarkable completeness and fidelity. We also demonstrate the utility of this uncertainty-aware approach by enhancing SOTA neural SLAM systems through an active ray sampling strategy. Extensive evaluations of NARUTO in various environments, using an indoor scene simulator, confirm its superior performance and state-of-the-art status in active reconstruction, as evidenced by its impressive results on benchmark datasets like Replica and MP3D.

Figures (25)

• Figure 1: We introduce a neural active reconstruction system, named NARUTO, which is guided by learned uncertainty. NARUTO enables an agent to identify areas of uncertainty and proactively investigate these regions to minimize reconstruction ambiguity. Consequently, this approach facilitates the incremental completion of the entire scene's reconstruction. NARUTO represents the first neural active Reconstruction system capable of functioning in large-scale environments with unrestricted movement.
• Figure 2: NARUTO framework Upon reaching a keyframe step, HabitatSim savva2019habitat generates posed RGB-D images. A select number of pixels from these images are sampled and stored in the observation database. Utilizing a mixed ray sampling strategy (combining Random and Active methods), a subset of rays is selected from the current keyframe and the database. These rays are then processed through the Hybrid Scene Representation (Map) to deduce the corresponding color, Signed Distance Function (SDF), depth, and uncertainty values. The predictions derived from this process facilitate uncertainty-aware bundle adjustment, updating both the scene's geometry and reconstruction uncertainty. Subsequently, the Map is refreshed, and our novel uncertainty-aware planning algorithm is employed to determine a goal and trajectory based on the SDFs and uncertainties. The agent then executes the planned action.
• Figure 3: Uncertainty-aware Planning Illustration. The top-k uncertain points are accumulated within the sensing range at each potential goal location. The goal with the greatest level of uncertainty is subsequently selected as the provisional target location. Efficient RRT planning effectively identifies a viable trajectory from the agent's current position to the designated goal.
• Figure 4: Matterport3D Results Two scenes (Left: pLe4; Right: HxpK) are presented here. The results are distinguished by border colors: [Ground Truth , ANMyan2023active, Ours]. In our results, notably in the second and fifth columns, black regions signify incomplete GT mesh, illustrating the extrapolation capacity of our neural mapping module. Results in columns 3 and 6 are trimmed for better comparison.
• Figure 5: Evolution of Uncertainty and Completion Using Explicit Grid and Implicit Net. The abrupt decrease in Grid Uncert(office3) correlates with the implementation of the reachability filtering strategy, as outlined in \ref{['sec:method:active_plan']}.