FPO++: Efficient Encoding and Rendering of Dynamic Neural Radiance Fields by Analyzing and Enhancing Fourier PlenOctrees

TL;DR

This work tackles artifacts in dynamic neural radiance fields encoded with Fourier PlenOctrees (FPO) by analyzing how Fourier compression interacts with volume rendering. It introduces a two-part density encoding—logarithmic encoding $e_{ ext{log}}(\sigma)=\,log(\sigma+1)$ and a component-dependent encoding $e_{ ext{comp}}(\sigma)$—to align the low-frequency representation with the transfer function and to counter underestimation under limited Fourier coefficients, alongside training-data augmentation to relax periodicity. The method preserves FPO’s compactness and differentiability, yielding improved geometric fidelity and color across synthetic and real scenes with real-time rendering, and shows substantial speedups in practice. These insights and encoding strategies may benefit other Fourier-based neural rendering approaches, enabling robust, fast4D scene representations in real-world deployments.

Abstract

Fourier PlenOctrees have shown to be an efficient representation for real-time rendering of dynamic Neural Radiance Fields (NeRF). Despite its many advantages, this method suffers from artifacts introduced by the involved compression when combining it with recent state-of-the-art techniques for training the static per-frame NeRF models. In this paper, we perform an in-depth analysis of these artifacts and leverage the resulting insights to propose an improved representation. In particular, we present a novel density encoding that adapts the Fourier-based compression to the characteristics of the transfer function used by the underlying volume rendering procedure and leads to a substantial reduction of artifacts in the dynamic model. Furthermore, we show an augmentation of the training data that relaxes the periodicity assumption of the compression. We demonstrate the effectiveness of our enhanced Fourier PlenOctrees in the scope of quantitative and qualitative evaluations on synthetic and real-world scenes.

Figures (6)

• Figure 1: Density over time of a single octree leaf. The leaf is marked in red in the respective images taken from the same view at different time steps $t$. Although the opacity is similar in the views, highly varying densities are observed over time, except for $t=16$ where there is empty space in the tree leaf.
• Figure 2: Two exemplary density functions over time (left) reconstructed with different number of coefficients $K_\sigma$. The original function of $T$ time steps is equal to its reconstruction with $K_\sigma = 119$ Fourier coefficients. The falloff of the marked peaks relative to its original value (right) is depending on $K_\sigma$ and follows the linear scaling function $s(K_\sigma)$.
• Figure 3: A density function and its reconstruction $\pi_{K_\sigma}$ using the DFT and IDFT with ${K_\sigma=31}$ (top left) and the same function and its reconstruction after applying our logarithmic encoding $e_{\log}$ (center left). Their full and compressed Fourier representations (top right, center right) show that a logarithmically scaled function contains less high-frequency information that gets lost during compression. Applying the transfer function to the reconstructions (bottom left) shows that the logarithmic version can better represent the original one.
• Figure 4: Reconstruction of a density function (Orig.) using only DFT and IDFT as proposed for FPOs wang2022fourier and additionally in combination with our component-dependent (comp.) and logarithmic (log.) encoding on top of the DFT and IDFT.
• Figure 5: Renderings of FPOs of different dynamic scenes without and with our logarithmic and component-dependent encoding after 10 epochs of fine-tuning.