Hypernetworks for Generalizable BRDF Representation

TL;DR

This work introduces a generalizable BRDF representation that reconstructs measured BRDFs from sparse, unstructured samples by conditioning a neural BRDF field with a set-encoder–driven hypernetwork. The architecture combines a set encoder, a hypernetwork decoder, and a hyponet to produce a continuous BRDF representation capable of estimating unseen materials and compressing densely sampled BRDF data into compact embeddings (e.g., 7D). It demonstrates superior reconstruction quality over baselines on MERL and RGL datasets, robust to varying sample counts, and enables BRDF editing through embedding interpolation. The approach reduces data capture requirements and supports efficient, interactive material manipulation, with future work targeting improved specular estimation and SVBRDF extensions.

Abstract

In this paper, we introduce a technique to estimate measured BRDFs from a sparse set of samples. Our approach offers accurate BRDF reconstructions that are generalizable to new materials. This opens the door to BDRF reconstructions from a variety of data sources. The success of our approach relies on the ability of hypernetworks to generate a robust representation of BRDFs and a set encoder that allows us to feed inputs of different sizes to the architecture. The set encoder and the hypernetwork also enable the compression of densely sampled BRDFs. We evaluate our technique both qualitatively and quantitatively on the well-known MERL dataset of 100 isotropic materials. Our approach accurately 1) estimates the BRDFs of unseen materials even for an extremely sparse sampling, 2) compresses the measured BRDFs into very small embeddings, e.g., 7D.

Figures (7)

• Figure 1: A room scene rendered with our reconstructed materials, including sparse reconstruction (table top and legs, door, door and picture frames, hinge), compression (two teapots on the left, door handle) and BRDF interpolation (right-most teapot). Scene courtesy of Benedikt Bitterli.
• Figure 2: During training, the set encoder and hypernetwork decoder are trained on a set of materials to predict the weights of hyponet (MLP) so that it can reconstruct the training set. The BRDF data is provided as a set of BRDF coordinates, $H_n,D_n$, and the corresponding reflectance values $f_r(H_n,D_n)$. To reconstruct a new material from a small set of BRDF reflectance samples, the trained set encoder and hypernetwork decoder are used to predict the weights of hyponet for the unknown material. Once those weights are known, we can query BRDF at any coordinates and for any new materials, conditioned on the embedding of their sampled BRDF values.
• Figure 3: t-SNE clustering of the test embeddings with different sample sizes, including $N=8, 40, 160, 4\,000, 40\,000, 640\,000$.
• Figure 4: Qualitative comparison results for reconstruction with small sample sizes. Thanks to the prior that our hypernetwork model learns for material appearance through training, it can accurately estimate the BRDFs of unseen materials and preserve the colors better than the baselines.
• Figure 5: Average PSNR, Delta E (CIE 2000), and SSIM results across different sample sizes.