Plenoxels: Radiance Fields without Neural Networks

TL;DR

Plenoxels demonstrate that photorealistic view synthesis can be achieved with an explicit, sparse voxel grid storing per-voxel opacity and spherical-harmonic color coefficients, avoiding neural networks entirely. By employing trilinear interpolation, a coarse-to-fine pruning strategy, differentiable volume rendering, and targeted regularization, the approach delivers NeRF-like quality with two orders of magnitude faster training on bounded, forward-facing, and 360° scenes. The work includes extensive ablations and real-data experiments, showing strong performance and practical rendering speeds (interactive rates when rendering) while remaining simple to implement and extend. This suggests that the differentiable volumetric renderer and simple priors, rather than neural networks per se, are the key drivers of high-quality volumetric reconstruction, with significant implications for scalable 3D reconstruction pipelines.

Abstract

We introduce Plenoxels (plenoptic voxels), a system for photorealistic view synthesis. Plenoxels represent a scene as a sparse 3D grid with spherical harmonics. This representation can be optimized from calibrated images via gradient methods and regularization without any neural components. On standard, benchmark tasks, Plenoxels are optimized two orders of magnitude faster than Neural Radiance Fields with no loss in visual quality.

Figures (15)

• Figure 1: Plenoxel: Plenoptic Volume Elements for fast optimization of radiance fields. We show that direct optimization of a fully explicit 3D model can match the rendering quality of modern neural based approaches such as NeRF while optimizing over two orders of magnitude faster.
• Figure 2: Overview of our sparse Plenoxel model. Given a set of images of an object or scene, we reconstruct a (a) sparse voxel ("Plenoxel") grid with density and spherical harmonic coefficients at each voxel. To render a ray, we (b) compute the color and opacity of each sample point via trilinear interpolation of the neighboring voxel coefficients. We integrate the color and opacity of these samples using (c) differentiable volume rendering, following the recent success of NeRF mildenhall2020nerf. The voxel coefficients can then be (d) optimized using the standard MSE reconstruction loss relative to the training images, along with a total variation regularizer.
• Figure 3: Ablation over TV regularization. Clear artifacts are visible in the forward-facing scenes without TV on both $\sigma$ and SH coefficients, although PSNR does not always reflect this.
• Figure 4: Gradient sparsity. The gradient becomes very sparse spatially within the first 12800 batches (one epoch for the synthetic scenes), with as few as 1% of the voxels updating per batch in the synthetic case. This enables efficient training via sparse parameter updates. The solid lines show the mean and the shaded regions show the full range of values among all scenes of each type.
• Figure 5: 1 minute, 20 seconds. Results on the synthetic scenes after 1 epoch of optimization, an average of 1 minute and 20 seconds.