Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction

Cheng Sun, Min Sun, Hwann-Tzong Chen

TL;DR

The paper tackles the slow training times of Neural Radiance Fields by proposing a direct voxel-grid optimization approach that models scene geometry with a dense density grid and uses a lightweight hybrid color representation. The core innovations are post-activation interpolation on voxel densities and two priors to prevent degenerate geometry, enabling NeRF-comparable quality with dramatically faster convergence—about 15 minutes per scene on a single GPU. The method employs a coarse-to-fine strategy, pruning unknown space via free-space knowledge and progressively scaling voxel grids, achieving substantial speedups while maintaining rendering quality across five inward-facing benchmarks. This work reduces the barrier to practical NeRF-like reconstruction, facilitating interactive applications and rapid per-scene synthesis without cross-scene pretraining or depth guidance. Overall, it demonstrates that explicit voxel-based density modeling, when enhanced with tailored activations and priors, can rival implicit NeRF representations in both speed and accuracy.

Abstract

We present a super-fast convergence approach to reconstructing the per-scene radiance field from a set of images that capture the scene with known poses. This task, which is often applied to novel view synthesis, is recently revolutionized by Neural Radiance Field (NeRF) for its state-of-the-art quality and flexibility. However, NeRF and its variants require a lengthy training time ranging from hours to days for a single scene. In contrast, our approach achieves NeRF-comparable quality and converges rapidly from scratch in less than 15 minutes with a single GPU. We adopt a representation consisting of a density voxel grid for scene geometry and a feature voxel grid with a shallow network for complex view-dependent appearance. Modeling with explicit and discretized volume representations is not new, but we propose two simple yet non-trivial techniques that contribute to fast convergence speed and high-quality output. First, we introduce the post-activation interpolation on voxel density, which is capable of producing sharp surfaces in lower grid resolution. Second, direct voxel density optimization is prone to suboptimal geometry solutions, so we robustify the optimization process by imposing several priors. Finally, evaluation on five inward-facing benchmarks shows that our method matches, if not surpasses, NeRF's quality, yet it only takes about 15 minutes to train from scratch for a new scene.

Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction

TL;DR

The paper tackles the slow training times of Neural Radiance Fields by proposing a direct voxel-grid optimization approach that models scene geometry with a dense density grid and uses a lightweight hybrid color representation. The core innovations are post-activation interpolation on voxel densities and two priors to prevent degenerate geometry, enabling NeRF-comparable quality with dramatically faster convergence—about 15 minutes per scene on a single GPU. The method employs a coarse-to-fine strategy, pruning unknown space via free-space knowledge and progressively scaling voxel grids, achieving substantial speedups while maintaining rendering quality across five inward-facing benchmarks. This work reduces the barrier to practical NeRF-like reconstruction, facilitating interactive applications and rapid per-scene synthesis without cross-scene pretraining or depth guidance. Overall, it demonstrates that explicit voxel-based density modeling, when enhanced with tailored activations and priors, can rival implicit NeRF representations in both speed and accuracy.

Abstract

We present a super-fast convergence approach to reconstructing the per-scene radiance field from a set of images that capture the scene with known poses. This task, which is often applied to novel view synthesis, is recently revolutionized by Neural Radiance Field (NeRF) for its state-of-the-art quality and flexibility. However, NeRF and its variants require a lengthy training time ranging from hours to days for a single scene. In contrast, our approach achieves NeRF-comparable quality and converges rapidly from scratch in less than 15 minutes with a single GPU. We adopt a representation consisting of a density voxel grid for scene geometry and a feature voxel grid with a shallow network for complex view-dependent appearance. Modeling with explicit and discretized volume representations is not new, but we propose two simple yet non-trivial techniques that contribute to fast convergence speed and high-quality output. First, we introduce the post-activation interpolation on voxel density, which is capable of producing sharp surfaces in lower grid resolution. Second, direct voxel density optimization is prone to suboptimal geometry solutions, so we robustify the optimization process by imposing several priors. Finally, evaluation on five inward-facing benchmarks shows that our method matches, if not surpasses, NeRF's quality, yet it only takes about 15 minutes to train from scratch for a new scene.
Paper Structure (65 sections, 33 equations, 17 figures, 18 tables)

This paper contains 65 sections, 33 equations, 17 figures, 18 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Representations for novel view synthesis.
  4. Neural radiance fields.
  5. Hybrid volumetric representations.
  6. Fast NeRF rendering.
  7. Fast NeRF convergence.
  8. Preliminaries
  9. Post-activated density voxel grid
  10. Voxel-grid representation.
  11. Density voxel grid for volume rendering.
  12. Sharp decision boundary via post-activation.
  13. Fast and direct voxel grid optimization
  14. Coarse geometry searching
  15. Coarse scene representation.
  16. ...and 50 more sections

Figures (17)

  • Figure 1: Super-fast convergence by our method. The key to our speedup is to optimize the volume density modeled in a dense voxel grid directly. Note that our method needs neither a conversion step from any trained implicit model (e.g., NeRF) nor a cross-scene pretraining, i.e., our voxel grid representation is directly and efficiently trained from scratch for each scene.
  • Figure 2: Approach overview. We first review NeRF in \ref{['sec:pre_nerf']}. In \ref{['sec:post_act_vox']}, we present a novel post-activated density voxel grid to support sharp surface modeling in lower grid resolutions. In \ref{['sec:approach']}, we show our approach to the reconstruction of radiance field with super-fast convergence, where we first find a coarse geometry in \ref{['ssec:coarse_learn']} and then reconstruct the fine details and view-dependent effects in \ref{['ssec:fine_learn']}.
  • Figure 3: A single grid cell with post-activation is capable of modeling sharp linear surfaces.Left: We depict the toy task for a 2D grid cell, where a grid cell is optimized for the linear surface (decision boundary) across it. Right: Each column shows an example task for three different methods. The results show that a single grid cell with post-activation (\ref{['eq:post_activation']}) is adequate to recover faithfully the linear surface. Conversely, pre-activation (\ref{['eq:pre_activation']}) and in-activation (\ref{['eq:in_activation']}) fail to accomplish the tasks as they can only fit into smooth results, and thus would require more grid cells to recover the surface detail. See supplementary material for the mathematical proof.
  • Figure 4: Toy example on image fitting. The target 2D image is binary to imitate the scenario that most of the 3D space is either occupied or free. The objective is to reconstruct the target image by a low-resolution 2D grid. In each optimization step, the tunable 2D grid is queried by interpolation with pre-activation (\ref{['eq:pre_activation']}), in-activation (\ref{['eq:in_activation']}), or post-activation (\ref{['eq:post_activation']}) to minimize the mean squared error to the target image. The result reveals that the post-activation can produce sharp boundaries even with low grid resolution (\ref{['fig:exdv_poc_a']}) and is much better than the other two under various grid resolutions (\ref{['fig:exdv_poc_b']}). This motivates us to model the 3D geometry directly via voxel grids with post-activation.
  • Figure 5:
  • ...and 12 more figures