Resolution Where It Counts: Hash-based GPU-Accelerated 3D Reconstruction via Variance-Adaptive Voxel Grids

TL;DR

This work introduces MrHash, a variance-adaptive, multi-resolution voxel grid implemented on a flat GPU hash table to enable real-time TSDF fusion and rendering for RGB-D and LiDAR data. By leveraging per-voxel TSDF statistics, a single-address-space grid, and GPU-accelerated quad-tree Gaussian Splatting, the approach achieves average $O(1)$ access, significant memory savings, and up to 13× speedups over fixed-resolution baselines while maintaining reconstruction fidelity. The method includes a multi-resolution Marching Cubes extension for meshing across resolution boundaries, a streaming policy to manage GPU memory, and an image-space quadtree to adapt splat density during rendering. Extensive experiments on diverse datasets demonstrate competitive accuracy with large memory and runtime benefits, and the open-source CUDA/C++ implementation supports real-time reconstruction and rendering in resource-constrained settings.

Abstract

Efficient and scalable 3D surface reconstruction from range data remains a core challenge in computer graphics and vision, particularly in real-time and resource-constrained scenarios. Traditional volumetric methods based on fixed-resolution voxel grids or hierarchical structures like octrees often suffer from memory inefficiency, computational overhead, and a lack of GPU support. We propose a novel variance-adaptive, multi-resolution voxel grid that dynamically adjusts voxel size based on the local variance of signed distance field (SDF) observations. Unlike prior multi-resolution approaches that rely on recursive octree structures, our method leverages a flat spatial hash table to store all voxel blocks, supporting constant-time access and full GPU parallelism. This design enables high memory efficiency and real-time scalability. We further demonstrate how our representation supports GPU-accelerated rendering through a parallel quad-tree structure for Gaussian Splatting, enabling effective control over splat density. Our open-source CUDA/C++ implementation achieves up to 13x speedup and 4x lower memory usage compared to fixed-resolution baselines, while maintaining on par results in terms of reconstruction accuracy, offering a practical and extensible solution for high-performance 3D reconstruction.

Figures (11)

• Figure 1: System Overview:MrHash takes the input data and, if necessary, allocates the voxel blocks associated with such 3D points and integrates them into a multi-resolution sparse voxel grid. The blocks whose variance is below a predefined threshold are downsampled and re-integrated into the voxel grid at a coarser resolution. Moreover, our pipeline can either extract the isosurface from the volumetric representation or render the surrounding environment.
• Figure 2: Illustration of grid representation to hash table mapping. The hash function maps the world points from integer world coordinates to the respective buckets of the hash-table $\bm{H}$. Every entry of the hash-table contains a pointer to a block in the specific heap of that resolution.
• Figure 3: Adopted merging strategy. a) Voxel block with low tsdf variance is reallocated at a coarser resolution (red). b) Voxel block with high tsdf variance remains unchanged (green).
• Figure 4: Ambiguity of sdf interpolation. The blue vertex attempts to interpolate the sdf value from its four neighboring voxels during sampling, but encounters undefined entries due to missing neighbors. To address this, we apply a weighted interpolation scheme that prioritizes finer-resolution values where available. For clarity, the visualization is shown in 2D assuming bilinear interpolation.
• Figure 5: Transitional voxels. (a) Due to the measurement-driven allocation, overlapping regions (dotted area) can occur between voxel blocks of different resolutions, which poses challenges during the vertex evaluation step in Marching Cubes. (b) To address this, we reduce the size of coarser voxels and adjust the placement of Marching Cubes vertices accordingly to maintain consistency across resolutions.