Rip-NeRF: Anti-aliasing Radiance Fields with Ripmap-Encoded Platonic Solids

TL;DR

Rip-NeRF introduces a novel anti-aliasing radiance field representation by projecting anisotropic 3D Gaussians onto the faces of Platonic solids (Platonic Solid Projection) and encoding those projections with a learnable Ripmap (Ripmap Encoding). This combination enables precise, efficient anisotropic area-sampling that surpasses isotropic and prior hybrid approaches, yielding state-of-the-art rendering quality on multi-scale synthetic data and real-world captures while maintaining reasonable training and memory requirements. The method offers a flexible trade-off between rendering quality and efficiency through the choice of Platonic solid and ripmap configuration, and its ablations demonstrate the complementary benefits of the two core components. Overall, Rip-NeRF provides a practical, scalable solution for high-fidelity anti-aliased neural radiance fields with strong performance on fine structural details and textures.

Abstract

Despite significant advancements in Neural Radiance Fields (NeRFs), the renderings may still suffer from aliasing and blurring artifacts, since it remains a fundamental challenge to effectively and efficiently characterize anisotropic areas induced by the cone-casting procedure. This paper introduces a Ripmap-Encoded Platonic Solid representation to precisely and efficiently featurize 3D anisotropic areas, achieving high-fidelity anti-aliasing renderings. Central to our approach are two key components: Platonic Solid Projection and Ripmap encoding. The Platonic Solid Projection factorizes the 3D space onto the unparalleled faces of a certain Platonic solid, such that the anisotropic 3D areas can be projected onto planes with distinguishable characterization. Meanwhile, each face of the Platonic solid is encoded by the Ripmap encoding, which is constructed by anisotropically pre-filtering a learnable feature grid, to enable featurzing the projected anisotropic areas both precisely and efficiently by the anisotropic area-sampling. Extensive experiments on both well-established synthetic datasets and a newly captured real-world dataset demonstrate that our Rip-NeRF attains state-of-the-art rendering quality, particularly excelling in the fine details of repetitive structures and textures, while maintaining relatively swift training times.

Figures (8)

• Figure 1: The two anisotropic areas $\text{Area}_1$ and $\text{Area}_2$ from different cones are ambiguously mapped to the same sampling area under the isotropic area-sampling (a), while are distinguishable under anisotropic area-sampling (b).
• Figure 2: Overview of our Rip-NeRF. We first cast a cone for each pixel, and then divide the cone into multiple conical frustums, which are further characterized by anisotropic 3D Gaussians parameterized by their mean and covariance $(\boldsymbol{\mu}, \; \boldsymbol{\Sigma})$. Next, to featurize a 3D Gaussian, we project it onto the unparalleled faces of the Platonic solid, denoted as $\{\mathcal{P}_i \; | \; i=1,...,n\}$ to form a 2D Gaussian $(\boldsymbol{\mu}_\text{proj}, \; \boldsymbol{\Sigma}_\text{proj})$, while the Platonic solid's faces are represented by the Ripmap Encoding with learnable parameters. Subsequently, we perform tetra-linear interpolation on the Ripmap Encoding to query corresponding feature vectors $f_i$ for the 2D Gaussian, where the position $(p_x, \; p_y)$ and level $(l_x, \; l_y)$ used in the interpolation are determined by the mean and covariance $(\boldsymbol{\mu}_\text{proj}, \; \boldsymbol{\Sigma}_\text{proj})$ of the 2D Gaussian, respectively. Finally, feature vectors $f_i$ from all Platonic solids' faces and the encoded view direction $d$ are aggregated together to estimate the color $c$ and density $\sigma$ of the conical frustums by a tiny MLP $F_\theta$.
• Figure 3: Comparison between isotropic (a) and anisotropic (b) area sampling in Mipmap and Ripmap, for characterizing the projected Gaussian.
• Figure 4: Two 3D ellipsoids, whose major axes are aligned along two different body diagonals of a cube, share the same 2D AABBs on the orthogonal tri-plane (a), making them indistinguishable under the Ripmap encoding. However, their difference can be captured by an additional different-oriented plane (b).
• Figure 5: Qualitative comparison of the full-resolution (close-up views) renderings on the multi-scale Blender dataset. PSNR/SSIM values are shown at the bottom of each result.