VoxNeuS: Enhancing Voxel-Based Neural Surface Reconstruction via Gradient Interpolation

TL;DR

VoxNeuS tackles gradient discontinuities in voxel-based SDF representations by substituting the analytical gradient from trilinear interpolation with an interpolated gradient, thereby stabilizing optimization and improving surface quality. It additionally enforces SDF regularization directly on grid vertices via explicit gradient computations, and employs a geometry-radiance disentangled architecture to prevent radiance optimization from deforming geometry. The method uses progressive SDF grid super-resolution and CUDA-accelerated primitives to achieve fast training (around 15 minutes) and low memory usage (~2–3 GB) on a single consumer GPU, while delivering superior reconstruction fidelity on DTU and BlendedMVS datasets compared to prior voxel-based and NeRF-like approaches. These contributions yield a practical, scalable pipeline for high-frequency, texture-aware 3D reconstruction with reduced computational overhead. The work demonstrates that gradient continuity and explicit regularization are crucial for efficient and accurate neural surface reconstruction in grid-based representations.

Abstract

Neural Surface Reconstruction learns a Signed Distance Field~(SDF) to reconstruct the 3D model from multi-view images. Previous works adopt voxel-based explicit representation to improve efficiency. However, they ignored the gradient instability of interpolation in the voxel grid, leading to degradation on convergence and smoothness. Besides, previous works entangled the optimization of geometry and radiance, which leads to the deformation of geometry to explain radiance, causing artifacts when reconstructing textured planes. In this work, we reveal that the instability of gradient comes from its discontinuity during trilinear interpolation, and propose to use the interpolated gradient instead of the original analytical gradient to eliminate the discontinuity. Based on gradient interpolation, we propose VoxNeuS, a lightweight surface reconstruction method for computational and memory efficient neural surface reconstruction. Thanks to the explicit representation, the gradient of regularization terms, i.e. Eikonal and curvature loss, are directly solved, avoiding computation and memory-access overhead. Further, VoxNeuS adopts a geometry-radiance disentangled architecture to handle the geometry deformation from radiance optimization. The experimental results show that VoxNeuS achieves better reconstruction quality than previous works. The entire training process takes 15 minutes and less than 3 GB of memory on a single 2080ti GPU.

Figures (9)

• Figure 1: Previous grid-based methods adopt trilinear interpolation to retrieve SDF values and analytically solve SDF gradient. However, the gradient discontinuity leads to glitches in volume rendering weights, thus degrading the reconstruction quality. Our proposed interpolated gradient eliminates the discontinuity and smooths the reconstructed surface without additional computation overhead.
• Figure 2: Reconstruction results from different methods. The running time and GPU memory consumption are tested on a single RTX 3090 for fair comparison, as 2080ti can't meet the memory requirement of Voxurfwu2022voxurf. (b) Our proposed VoxNeuS replaces analytical gradient with interpolated gradient to handle the gradient discontinuity in trilinear interpolation. Combined with a simple network structure, our method achieves efficient and high-quality reconstruction. (c) When the interpolation gradient is removed and other structures remain unchanged, the reconstruction quality drops drastically. (d) Voxurf adopts dual color network and other tricks to ease the optimization of the SDF grid, which leads to some artifacts on the surface.
• Figure 3: Gradient continuity on junction surface. A point ${\boldsymbol{x}}$ lies on the junction surface of two adjacent cubes. $V_a$ is the set of points on the junction surface while $V_b$ and $V_b'$ are the sets of points outside the junction surface. As for analytical gradient at ${\boldsymbol{x}}$, the $grad^{(a)}_r$ from cube $V_a\bigcup V_b$ is not agree with the $grad^{(a)}_l$ from cube $V_a\bigcup V_b'$, showing discontinuity. As for interpolated gradient at ${\boldsymbol{x}}$, as the weight of $V_b$ and $V_b'$ is zero, the $grad^{(i)}_r$ is agree with $grad^{(i)}_l$, showing continuity.
• Figure 4: An 2D example to illustrate the continuity of the analytical and interpolated gradient. In this example, a ray is marching to the curved surface. The ground truth SDF values are stored on the vertices of the dense grid. The analytical gradient is computed with Eq. \ref{['eq:grada']} and the interpolated gradient is computed with Eq. \ref{['eq:gradi1']}. The discontinuity of the analytical gradient can be observed at the junction surface (left). Such discontinuity causes glitches on $cos\langle {\boldsymbol{n}}, {\boldsymbol{v}}\rangle$, opacity $\alpha$, and volume rendering weights $w$ (right). After applying an interpolated gradient, the estimated gradient gets close to ground truth and the glitches are eliminated.
• Figure 5: Network architecture overview.