Sharpening Your Density Fields: Spiking Neuron Aided Fast Geometry Learning

TL;DR

This work tackles robust geometry extraction in grid-based NeRF by removing the need for manual density thresholds through a learnable spiking-threshold mechanism. A round-robin training strategy alternates between normal and spiking stages to stabilize optimization and sharpen the density distribution with minimal computational overhead. Empirical results on Blender and DTU demonstrate improved geometry quality and competitive efficiency compared to state-of-the-art baselines, with ablations confirming the necessity of the round-robin and fixed-color components. The approach broadens practical applicability of NeRF geometry extraction, including challenging scenarios such as transparent objects, by enabling fast, high-fidelity surface reconstruction without manual tuning.

Abstract

Neural Radiance Fields (NeRF) have achieved remarkable progress in neural rendering. Extracting geometry from NeRF typically relies on the Marching Cubes algorithm, which uses a hand-crafted threshold to define the level set. However, this threshold-based approach requires laborious and scenario-specific tuning, limiting its practicality for real-world applications. In this work, we seek to enhance the efficiency of this method during the training time. To this end, we introduce a spiking neuron mechanism that dynamically adjusts the threshold, eliminating the need for manual selection. Despite its promise, directly training with the spiking neuron often results in model collapse and noisy outputs. To overcome these challenges, we propose a round-robin strategy that stabilizes the training process and enables the geometry network to achieve a sharper and more precise density distribution with minimal computational overhead. We validate our approach through extensive experiments on both synthetic and real-world datasets. The results show that our method significantly improves the performance of threshold-based techniques, offering a more robust and efficient solution for NeRF geometry extraction.

Figures (12)

• Figure 1: Visual comparisons of our method and Nerfacto tancik2023nerfstudio. Both models are trained for 10,000 iterations within 5 minutes, without the mask loss. As shown in the rendered color images and normal maps, our method extracts cleaner and more accurate geometry, while Nerfacto exhibits floating artifacts in the surrounding areas.
• Figure 2: Framework overview of our method. The top of this figure is the pipeline, which depicts the network architecture with the round-robin strategy. The blue region represents the normal stage and the red region is the spiking stage. The bottom of the figure illustrates the loss computation process. The orange region means density values participate in volume rendering.
• Figure 3: Visual Quality Comparisons on the Blender dataset mildenhall2020nerf and the DTU dataset jensen2014large. Red bounding boxes are used to highlight areas with significant differences. Our method can reconstruct more high-quality geometries. The full set of visual comparisons is provided in the supplementary materials.
• Figure 4: Ablation studies on the DTU dataset jensen2014large and the Blender dataset mildenhall2020nerf.
• Figure 5: Reconstruction results on the Blender dataset mildenhall2020nerf (Chair, Drums).