Uncertainty for SVBRDF Acquisition using Frequency Analysis

TL;DR

This work tackles the challenge of uncertainty in passive SVBRDF acquisition under uncontrolled lighting and unstructured viewpoints by introducing a fast, entropy-based uncertainty map computed in the spherical-harmonics frequency domain. By reformulating the reflection model as a convolution and leveraging the power spectrum of spherical-harmonic coefficients, the authors achieve millisecond-scale uncertainty estimates and competitive SVBRDF reconstruction performance. They validate the method against physically based path tracing, show a strong correlation between entropy and reconstruction error, and demonstrate practical applications such as capture guidance, information sharing, and diffusion-based inpainting, along with initialization speedups for optimization. These contributions enable robust, real-world SVBRDF capture workflows with significantly reduced computation and actionable guidance for data acquisition and post-processing.

Abstract

This paper aims to quantify uncertainty for SVBRDF acquisition in multi-view captures. Under uncontrolled illumination and unstructured viewpoints, there is no guarantee that the observations contain enough information to reconstruct the appearance properties of a captured object. We study this ambiguity, or uncertainty, using entropy and accelerate the analysis by using the frequency domain, rather than the domain of incoming and outgoing viewing angles. The result is a method that computes a map of uncertainty over an entire object within a millisecond. We find that the frequency model allows us to recover SVBRDF parameters with competitive performance, that the accelerated entropy computation matches results with a physically-based path tracer, and that there is a positive correlation between error and uncertainty. We then show that the uncertainty map can be applied to improve SVBRDF acquisition using capture guidance, sharing information on the surface, and using a diffusion model to inpaint uncertain regions. Our code is available at https://github.com/rubenwiersma/svbrdf_uncertainty.

Figures (10)

• Figure 1: Passive capture can lead to ambiguity about the material of an object. Which plane is glossy and which is matte? While it is hard to see in the left scene, the materials are the same in both scenes (left matte, right glossy).
• Figure 2: An overview of our pipeline showing input, output, and the proposed algorithm. The input to our approach is a set of photographs of an object from multiple viewpoints and the associated camera extrinsics and intrinsics. We also provide the input lighting as an environment map and object geometry. We first estimate spherical harmonic coefficients on both the incoming and outgoing radiance and then estimate the effect of different BRDF filters within the power spectrum. This is used to compute a measure of uncertainty for the predicted parameters of the acquisition.
• Figure 3: Examples of the likelihood and entropy $H$ for several parameter/lighting combinations. Left: An ideal situation, where the lighting is a dirac delta. Center: A situation where the lighting is too low-frequency to recover a good $\alpha$ value. Right: A situation where the incoming radiance is ideal, but the specular component is too low to get a proper recovery for $\alpha$.
• Figure 4: Material recovery results on Stanford ORB for Mitsuba ($52.49$s on average), the SH Model and our proposed global sharing application ($5.07$s on average).
• Figure 5: Material recovery results on the synthetic dataset for Mitsuba, the SH Model, and the SH Model with global sharing enabled by entropy. We see that global sharing helps smooth and improve the results obtained with the SH Model without blurring the maps.