H3-Mapping: Quasi-Heterogeneous Feature Grids for Real-time Dense Mapping Using Hierarchical Hybrid Representation

TL;DR

The paper addresses the challenge of real-time dense mapping with high texture fidelity in NeRF-based approaches, where texture modeling is a bottleneck. It introduces H3-Mapping, combining quasi-heterogeneous feature grids with a gradient-aided coverage-maximizing keyframe policy and a hierarchical hybrid representation that splits geometry priors and texture modeling across efficient data structures. Key contributions include space-warped quasi grids tailored to texture complexity, a multi-hash grid scheme to handle warped spaces, a gradient-guided keyframe strategy with adaptive pruning, and SDF-based volume rendering with a multi-loss optimization framework. Empirical results across Replica and ScanNet show superior texture fidelity, improved geometry accuracy, and faster runtimes compared to leading baselines, demonstrating practical viability for robotics, AR/VR, and digital twins; code is publicly available.

Abstract

In recent years, implicit online dense mapping methods have achieved high-quality reconstruction results, showcasing great potential in robotics, AR/VR, and digital twins applications. However, existing methods struggle with slow texture modeling which limits their real-time performance. To address these limitations, we propose a NeRF-based dense mapping method that enables faster and higher-quality reconstruction. To improve texture modeling, we introduce quasi-heterogeneous feature grids, which inherit the fast querying ability of uniform feature grids while adapting to varying levels of texture complexity. Besides, we present a gradient-aided coverage-maximizing strategy for keyframe selection that enables the selected keyframes to exhibit a closer focus on rich-textured regions and a broader scope for weak-textured areas. Experimental results demonstrate that our method surpasses existing NeRF-based approaches in texture fidelity, geometry accuracy, and time consumption. The code for our method will be available at: https://github.com/SYSU-STAR/H3-Mapping.

Figures (12)

• Figure 1: Mapping results of two recent NeRF-based dense mapping methodsjiang2023h2johari2022eslam(a,b) and our proposed method(c) on Room1 in the Replica datasetstraub2019replica with varying training iterations per frame. It reveals that the geometry error (measured as accuracy(Acc.) and completion(Comp.)) reaches a low value with limited iterations. In methodsjiang2023h2johari2022eslam, the texture (evaluated by PSNR) requires further training, but our method achieves higher PSNR with fewer iterations by rapidly converging. The metrics are defined in Sec.\ref{['subsubsec:metrics']}.
• Figure 2: The illustration of quasi-heterogeneous feature grids.
• Figure 3: The illustration of the quasi-heterogeneous feature grids. For simplicity, we assume two maximum low-frequency directions ($M = 2$) and only show the features of one level in the multiresolution hash grids.
• Figure 4: Illustration of the validation process for a line segment. Red points: Uniformly sampled points along the line connecting the two detected endpoints. Blue points: Back-projected points in the direction of the red points.
• Figure 5: To find the low-frequency directions, we fuse the line directions of line segments that are similar to the one with the highest score in each frame and combine the results from multiple frames to obtain the updated directions.