Evaluate Geometry of Radiance Fields with Low-frequency Color Prior

TL;DR

The paper tackles the problem of evaluating radiance-field geometry without ground-truth, introducing Inverse Mean Residual Color (IMRC) as a geometry metric based on a low-frequency color prior. IMRC estimates a residual color field from observation images by approximating colors with low-frequency spherical harmonics and weighting by transmittance, then reports IMRC = -10 · log10(MRC) to quantify geometry quality. Through experiments on DTU, NeRF Synthetic, and LLFF, IMRC demonstrates alignment with geometry quality and complements image-based metrics, enabling ground-truth-free benchmarking of radiance-field methods. The work provides a practical tool to assess and compare the geometry of neural radiance fields and encourages further exploration of geometry-centric evaluation in 3D reconstruction and view synthesis.

Abstract

A radiance field is an effective representation of 3D scenes, which has been widely adopted in novel-view synthesis and 3D reconstruction. It is still an open and challenging problem to evaluate the geometry, i.e., the density field, as the ground-truth is almost impossible to obtain. One alternative indirect solution is to transform the density field into a point-cloud and compute its Chamfer Distance with the scanned ground-truth. However, many widely-used datasets have no point-cloud ground-truth since the scanning process along with the equipment is expensive and complicated. To this end, we propose a novel metric, named Inverse Mean Residual Color (IMRC), which can evaluate the geometry only with the observation images. Our key insight is that the better the geometry, the lower-frequency the computed color field. From this insight, given a reconstructed density field and observation images, we design a closed-form method to approximate the color field with low-frequency spherical harmonics, and compute the inverse mean residual color. Then the higher the IMRC, the better the geometry. Qualitative and quantitative experimental results verify the effectiveness of our proposed IMRC metric. We also benchmark several state-of-the-art methods using IMRC to promote future related research. Our code is available at https://github.com/qihangGH/IMRC.

Figures (20)

• Figure 1: One example of novel-view images, rendered depth from reconstructed density fields, and residual color of TensoRF AnpeiChen2022TensoRFTR, DVGO ChengSun2022DVGO, and DVGO ChengSun2022DVGO + Distortion Loss barron2022mipnerf360. From top two rows, although DVGO achieves a better PSNR$\uparrow$, its geometry is qualitatively worse than TensoRF. From the bottom row, distortion loss could be qualitatively verified to improve the geometry. Our IMRC$\uparrow$ quantitatively evaluates these results correctly.
• Figure 2: The color frequency tends to be lower and lower when the point approaches a surface.
• Figure 3: As the points $\textbf{v}_1$ and $\textbf{v}_2$ demonstrate, inaccurate surfaces lie inside or outside the ground-truth surface will lead to higher-frequency colors.
• Figure 4: Residual color computation. (a) demonstrates an example scene and its reconstructed density field. For a point $\textbf{v}$, we can obtain its observation colors (b) according to the captured images. To tackle the occlusion, the transmittance between the point $\textbf{v}$ and each camera $C$ could be taken as the confidence of each observation, i.e., (c). Based on (b) and (c), the color could be weighted-transformed (d) into frequency domain (f) with the residual color (e). Due to the confidence (c), the residual color will be large if its corresponding confidence is low. To this end, the final residual color (g) should be weighted by observation confidence (c).
• Figure 5: Examples of the CD$\downarrow$/IMRC$\uparrow$/UserRank$\downarrow$ consistent results on the DTU dataset. For each example, left is better. From left to right, Scan 114, 65, and 40.