On Neural BRDFs: A Thorough Comparison of State-of-the-Art Approaches

TL;DR

This work systematically compares neural BRDF approaches, spanning parametric-model-based neural methods and purely neural BRDFs, under fixed geometry and calibrated lighting to isolate reflectance modeling. It introduces a reciprocity-enforcing input mapping and an enhanced additive-split strategy to improve physical fidelity, alongside a thorough evaluation on both semi-synthetic MERL-based data and real-world DiLiGenT-MV data. The findings show purely neural BRDFs excel for highly specular materials in synthetic settings, while differences on real data are smaller; importantly, reciprocity and energy conservation are not reliably learned from data alone, motivating the proposed constraints. These insights guide practical choices for neural BRDFs and provide concrete techniques to enforce fundamental physical properties in neural rendering pipelines.

Abstract

The bidirectional reflectance distribution function (BRDF) is an essential tool to capture the complex interaction of light and matter. Recently, several works have employed neural methods for BRDF modeling, following various strategies, ranging from utilizing existing parametric models to purely neural parametrizations. While all methods yield impressive results, a comprehensive comparison of the different approaches is missing in the literature. In this work, we present a thorough evaluation of several approaches, including results for qualitative and quantitative reconstruction quality and an analysis of reciprocity and energy conservation. Moreover, we propose two extensions that can be added to existing approaches: A novel additive combination strategy for neural BRDFs that split the reflectance into a diffuse and a specular part, and an input mapping that ensures reciprocity exactly by construction, while previous approaches only ensure it by soft constraints.

Figures (17)

• Figure 1: In this work, we present a thorough comparison of neural approaches for BRDF parametrization, including methods based on parametric models like the Disney BRDF burley2012physically and purely neural approaches. We show that while purely neural approaches have advantages for highly specular materials (top row: semi-synthetic, chrome steel from MERL matusik2003MERL), we find much less difference between the methods for the DiLiGenT-MV real-world dataset Li2020DiLiGentMVDataset (bottom row: pot2 from DiLiGenT-MV).
• Figure 2: Overview of the neural models to compute the BRDF $f(x, l, v)$. Approaches based on a parametric model employ an MLP to predict the parameters for the respective model. A sigmoid or softplus output nonlinearity is used, depending on the range of the parameters. Methods based on a single MLP compute the BRDF value directly from the position and the view and light direction. Additive models split the reflection into a diffuse and a specular component. While the separate architecture uses independent MLPs for both, the shared architecture uses a common MLP with two separate heads. For all purely neural approaches, the Rusinkiewicz angles rusinkiewicz1998new are used to parametrize the directions. Moreover, we employ an intrinsic approach KoestlerIntrinsicNeuralFields22 to encode the position directly on the mesh as well as positional encoding for the angles. See \ref{['sec:angleAndEncoding']} for details on the angles and the encoding.
• Figure 3: Qualitative evaluation of the reconstruction for the MERL grease covered steel BRDF matusik2003MERL uniformly rendered on the bunny mesh from jacobson2020common and the cow object from the real-world DiLiGenT-MV data Li2020DiLiGentMVDataset. Shown are renderings in sRGB space with the corresponding PSNR values and the FLIP error maps for the sRGB renderings. Only the purely neural approaches ($$∎) are able to reconstruct the intricate reflection patterns for the grease covered steal faithfully. For the cow, the results for the parametric models ($$∎) are much more on par, and all models show difficulties in similar areas -- in particular in recesses, which suggests unmodeled interreflections as a potential reason.
• Figure A.1: Visualization of the view-light and the Rusinkiewicz angles on the left and on the right, respectively. Shown are view and light direction, $v$ and $l$ in a local coordinate system given by surface normal $n$, a surface tangent $t$ (which is arbitrary in the case of isotropic BRDFs) and the binormal $b=n\times t$. The view-light angles are simply the polar angles of $v$ and $l$, while the Rusinkiewicz angles are given in terms of the half angle $h=\frac{v + l}{\|v + l\|}$ and a "difference" vector $d$ which is the light direction $l$ in a coordinate system with $h$ as the normal.
• Figure C.1: Qualitative BRDF reconstruction for the cow object from the DiLiGenT-MV dataset Li2020DiLiGentMVDataset for the additive separate architecture with a scalar specular term (as suggested in NeRFactor Zhang2021NeRFactor). The results show, that a scalar specular term is unable to reconstruct the reflectance of this object and creates a spurious glow. This indicates that for some materials, a specular term with 3 channels is necessary to yield high-quality reconstructions.