SPC-NeRF: Spatial Predictive Compression for Voxel Based Radiance Field

TL;DR

Inspired by the prosperous digital image compression techniques, this paper proposes SPC-NeRF, a novel framework applying spatial predictive coding in EVG NeRF compression that can remove spatial redundancy efficiently for better compression performance.

Abstract

Representing the Neural Radiance Field (NeRF) with the explicit voxel grid (EVG) is a promising direction for improving NeRFs. However, the EVG representation is not efficient for storage and transmission because of the terrific memory cost. Current methods for compressing EVG mainly inherit the methods designed for neural network compression, such as pruning and quantization, which do not take full advantage of the spatial correlation of voxels. Inspired by prosperous digital image compression techniques, this paper proposes SPC-NeRF, a novel framework applying spatial predictive coding in EVG compression. The proposed framework can remove spatial redundancy efficiently for better compression performance.Moreover, we model the bitrate and design a novel form of the loss function, where we can jointly optimize compression ratio and distortion to achieve higher coding efficiency. Extensive experiments demonstrate that our method can achieve 32% bit saving compared to the state-of-the-art method VQRF on multiple representative test datasets, with comparable training time.

Figures (8)

• Figure 1: Rate-Distortion curves of our method compared to state-of-the-art EVG NeRF compression methods on the Synthetic NeRF dataset.
• Figure 2: Two different approaches to mapping 3D coordinates to color value $c$ and volume density $\sigma$. $\bf{x}$ denotes the input Cartesian coordinate, $\Phi$ and $\Psi$ denotes the 3D coordinate and the view direction of positional encoding, respectively. $F^c$ represents the color feature.
• Figure 3: The overview framework of our method. The orange font indicates the variables necessary to reconstruct the feature grid, i.e., the syntax elements that are written into the coding bitstream.
• Figure 4: The yellow voxel denotes the current processing voxel; others are the available neighbors, wherein the green voxels represent the reference candidates we used with their indexes, i.e., voxels from which we select reference.
• Figure 5: Distribution prediction residuals in synthetic lego (quantization step = 0.5).