NerfAcc: Efficient Sampling Accelerates NeRFs

TL;DR

<3-5 sentence high-level summary> Neural Radiance Fields renderings are computationally expensive due to dense sampling along rays. The authors propose NerfAcc, a plug-and-play Python toolbox that treats sampling as a transmittance-estimation problem, decoupling it from the neural radiance field and enabling speedups of 1.5x to 20x across diverse NeRF variants, including Instant-NGP implemented in pure Python. They unify existing sampling strategies under a single transmittance framework, analyze design choices (uniform, spatial skipping, PDF approaches), and demonstrate how occupancy grids and proposal networks can be combined. The work contributes both theoretical insight into sampling as transmittance estimation and a practical tool that accelerates NeRF research and broadens accessibility.

Abstract

Optimizing and rendering Neural Radiance Fields is computationally expensive due to the vast number of samples required by volume rendering. Recent works have included alternative sampling approaches to help accelerate their methods, however, they are often not the focus of the work. In this paper, we investigate and compare multiple sampling approaches and demonstrate that improved sampling is generally applicable across NeRF variants under an unified concept of transmittance estimator. To facilitate future experiments, we develop NerfAcc, a Python toolbox that provides flexible APIs for incorporating advanced sampling methods into NeRF related methods. We demonstrate its flexibility by showing that it can reduce the training time of several recent NeRF methods by 1.5x to 20x with minimal modifications to the existing codebase. Additionally, highly customized NeRFs, such as Instant-NGP, can be implemented in native PyTorch using NerfAcc.

Figures (6)

• Figure 1: NerfAcc Toolbox. Our proposed toolbox, NerfAcc, integrates advanced efficient sampling techniques that lead to significant speedups in training various recent NeRF papers with minimal modifications to existing codebases.
• Figure 2: Illustration of Sampling via Transmittance Estimator. Although spatial skipping approaches (e.g., Occupancy Grid in Instant-NGP muller2022instant) and PDF approaches (e.g., Proposal Network in Mip-NeRF 360 barron2022mip360) appear distinct from each other, they can both be viewed as constructing a transmittance estimator from which samples can be created via importance sampling.
• Figure 3: Qualitative Results. NerfAcc is able to significantly reduce the training time of various NeRF-related methods across multiple datasets, while also yielding slightly improved quality in the majority of cases. The training time and test PSNR are shown on the left corner of each image.
• Figure 4: Plug-and-play Example in BARF lin2021barf's Repository. With around $50$ lines of code change, both training speed and performance can be greatly improved with efficient sampling equipped in NerfAcc, as shown in Table \ref{['tab:result_dynamic']}.
• Figure 5: Comparison between Different Sampling Approaches in terms of Training Time and Rendering FPS. We sweep the hyper-parameters space for each sampling approach, and find out that occupancy grid approach gives the best performance-speed trade-off on the NeRF-Synthetic dataset, while the proposal network approach performs the best on the Mip-NeRF 360 dataset. Note T (min) denotes for training time and FPS is for rendering frames per second. All experiments use the HashEncoding from Instant-NGP muller2022instant as the radiance field representation. Please see the supplementary materials for the hyperparameter space that we explored.