Acquisition of Spatially-Varying Reflectance and Surface Normals via Polarized Reflectance Fields

TL;DR

The paper tackles the challenging problem of simultaneously acquiring object geometry and spatially varying reflectance under complex, real-world lighting. It introduces polarized reflectance fields captured with a multi-view OLAT setup, and a three-stage pipeline (data preprocessing, initialization, optimization) to robustly separate diffuse/specular components and recover per-pixel normals and SVBRDF parameters. Key contributions include a practical polarized capture workflow, artifact-reduction strategies (overexposure, inter-reflection, lens flare, occlusion), and a rigorous optimization framework that yields accurate $\rho_d$, $\rho_s$, $n_d$, $n_s$, $\varsigma$, $\gamma$, and $\sigma$, validated via relighting and qualitative comparisons. The work enables realistic rendering and provides a valuable training dataset for inverse rendering, with potential extensions to neural methods for handling challenging inter-reflection scenarios and moving subjects.

Abstract

Accurately measuring the geometry and spatially-varying reflectance of real-world objects is a complex task due to their intricate shapes formed by concave features, hollow engravings and diverse surfaces, resulting in inter-reflection and occlusion when photographed. Moreover, issues like lens flare and overexposure can arise from interference from secondary reflections and limitations of hardware even in professional studios. In this paper, we propose a novel approach using polarized reflectance field capture and a comprehensive statistical analysis algorithm to obtain highly accurate surface normals (within 0.1mm/px) and spatially-varying reflectance data, including albedo, specular separation, roughness, and anisotropy parameters for realistic rendering and analysis. Our algorithm removes image artifacts via analytical modeling and further employs both an initial step and an optimization step computed on the whole image collection to further enhance the precision of per-pixel surface reflectance and normal measurement. We showcase the captured shapes and reflectance of diverse objects with a wide material range, spanning from highly diffuse to highly glossy - a challenge unaddressed by prior techniques. Our approach enhances downstream applications by offering precise measurements for realistic rendering and provides a valuable training dataset for emerging research in inverse rendering. We will release the polarized reflectance fields of several captured objects with this work.

Figures (21)

• Figure 1: Polarized OLAT. We present an object captured under the first 300 lighting conditions. The top two rows exhibit images captured through cross-polarized OLAT and parallel-polarized OLAT. In the last row, we zoom in on details, marked by corresponding continuous/dotted boxes, with the active light board in the lower corner accordingly. Additionally, we present the lighting order, progressing from blue to red.
• Figure 2: Overexposure Removal. We demonstrate the effectiveness of overexposure elimination by using a mirrorball. The observation under all-white lighting is a summation of all frames from the cross-polarized OLAT data $\Lambda_\perp$. We select points a and c from the zoom-in region and b from the base, which is made of relatively diffuse material. The second row presents the corresponding intensities recorded under cross-polarized OLAT lighting. The horizontal axis represents the OLAT index, while the vertical axis indicates the recorded intensities in red, green, and blue.
• Figure 3: Inter-reflection and Lens Flare. In row 1), we present the captured intensity distribution of a fixed surface point, indicated with a cross ($\mathbf{\times}$) in the following example images. The intensity patterns are identified as a) interreflection, b) regular specular reflection, and c) lens flare. Row 2) provides a false-color view, with a zoom-in view in the last row 3), as well as the raw capture. In the false-color view, the intensity strength is represented by the color, with stronger intensities appearing redder.
• Figure 4: Acquisition. We present a measured mug comprising a diffuse base and a clear coat showcasing a1) original, b1) diffuse albedo $\rho_d$, c1) specular albedo $\rho_s$, a2) diffuse normal $n_d$, b2) diffuse inter-reflection $\varrho_d$ (intensity adjusted for better visualization), c2) diffuse occlusion $\tau_d$, a3) specular normal $n_s$, b3) specular inter-reflection $\varrho_s$, c3) specular occlusion $\tau_d$, a4) specular variance $\sigma$, b4) anisotropy $\varsigma$, and c4) roughness $\gamma$.
• Figure 5: Normal Fusion. a1) original image, b1) maximum cross-correlation of diffuse normal optimization, c1) maximum cross-correlation of specular normal optimization, a2) fused normal, b2) diffuse normal, c2) specular normal.