Radiance Surfaces: Optimizing Surface Representations with a 5D Radiance Field Loss

TL;DR

The paper tackles the challenge of reconstructing surfaces from photographs by moving away from image-space losses on volumetric renders and instead optimizing a distribution over surface primitives through a radiance-field loss. By projecting training images into the scene and treating each potential surface patch as an independent perturbation, the method forms a spatio-directional loss that yields explicit surfaces suitable for level-set extraction, without heavy changes to existing NeRF-like pipelines. A stochastic background surface and non-local perturbations enable robust convergence and topological flexibility, while a volume-relaxation option preserves speed and quality in difficult regions. Empirically, the approach achieves novel-view quality comparable to NeRF with substantial rendering speedups and provides reliable geometry reconstruction, supported by demonstrations on indoor/outdoor datasets and standard benchmarks, along with a discussion of limitations and future directions. The framework promises a simple, fast, surface-based alternative to exponential-volume methods with practical benefits for editing, animation, and efficient rendering.

Abstract

We present a fast and simple technique to convert images into a radiance surface-based scene representation. Building on existing radiance volume reconstruction algorithms, we introduce a subtle yet impactful modification of the loss function requiring changes to only a few lines of code: instead of integrating the radiance field along rays and supervising the resulting images, we project the training images into the scene to directly supervise the spatio-directional radiance field. The primary outcome of this change is the complete removal of alpha blending and ray marching from the image formation model, instead moving these steps into the loss computation. In addition to promoting convergence to surfaces, this formulation assigns explicit semantic meaning to 2D subsets of the radiance field, turning them into well-defined radiance surfaces. We finally extract a level set from this representation, which results in a high-quality radiance surface model. Our method retains much of the speed and quality of the baseline algorithm. For instance, a suitably modified variant of Instant NGP maintains comparable computational efficiency, while achieving an average PSNR that is only 0.1 dB lower. Most importantly, our method generates explicit surfaces in place of an exponential volume, doing so with a level of simplicity not seen in prior work.

Figures (19)

• Figure 1: Comparison of the loss in volumetric optimization and our radiance field loss. We denote alpha blending by $\mathcal{R}$ and the color difference metric as ${\color{vecred}\ell}(L) \coloneq {\color{vecred}\ell}(L, L_\text{target})$ and drop its dependency on the target color for simplicity. Traditional volumetric reconstruction minimizes the image-space loss of blended colors. In contrast, our method minimizes a blended radiance field loss that yields a distribution of surfaces out of which a surface representation can be trivially extracted, e.g., via marching cubes.
• Figure 2: Non-local perturbations. We consider a single candidate surface patch (with color $L_\mathbf{p}$) along the ray as a perturbation of a background surface (with color $L_\mathbf{b}$). (a) Blending colors violates the surface assumption and leads to volumetric results. (b) We instead treat the perturbation as a random binary choice and optimize the associated discrete probability. The final reconstruction is non-random and will never blend the contribution of multiple surfaces.
• Figure 3: Surface candidates as independent subproblems. With multiple candidates along a ray, each perturbation is treated as an independent subproblem, resulting in local losses distributed spatially over the scene.
• Figure 4: Stochastic background. Selecting the background surface at random from a distribution $f_\mathbf{b}$ enables visibility through high-occupancy regions. Each sampled background surface defines a new perturbation problem solvable with the radiance field loss. Taking an expectation of this process leads to a simple deterministic expression that we implement in practice.
• Figure 5: Regularization. For simple scenes with enough observations, the reconstructed surface closely matches the ground truth geometry without requiring additional constraints. Adding Laplacian refinement helps smooth out unnecessary small kinks, resulting in a more accurate final geometry.