A Bayesian Inference Framework for Procedural Material Parameter Estimation

TL;DR

This work tackles inverse rendering for procedural material parameters from a single image. It introduces a Bayesian framework that couples priors over parameters with a Gaussian likelihood on image-summary differences, and allows both continuous and discrete parameters to be inferred. Point estimates are provided via MAP, while full posterior sampling is achieved with MCMC methods including Metropolis-Hastings, HMC, and MALA, enabling multiple plausible parameter solutions and uncertainty awareness. Demonstrations across wall plaster, leather, wood, anisotropic brushed metals, and layered metallic paints show the method’s ability to recover interpretable material parameters and to compare forward models, with practical benefits for editing and synthesis.

Abstract

Procedural material models have been gaining traction in many applications thanks to their flexibility, compactness, and easy editability. We explore the inverse rendering problem of procedural material parameter estimation from photographs, presenting a unified view of the problem in a Bayesian framework. In addition to computing point estimates of the parameters by optimization, our framework uses a Markov Chain Monte Carlo approach to sample the space of plausible material parameters, providing a collection of plausible matches that a user can choose from, and efficiently handling both discrete and continuous model parameters. To demonstrate the effectiveness of our framework, we fit procedural models of a range of materials---wall plaster, leather, wood, anisotropic brushed metals and layered metallic paints---to both synthetic and real target images.

Figures (10)

• Figure 3: A motivating example of a scattering material with two estimated parameters (scattering coefficient and phase function parameter). The posterior distribution sampled with our method for three synthetic input images is able to detect the full structure of the parameter space, matching the predictions from similarity theory.
• Figure 4: Results of our MCMC sampling on synthetic inputs. Each row corresponds to two examples of a different material model. For each example, the first column is the synthetic target image. We show MCMC samples in the other columns, where sample-1 and sample-2 are chosen closer to the peak of the posterior distribution, and sample-3 is further away. More results please refer to supplemental materials.
• Figure 5: MCMC sampling with discrete parameters. In these examples, we illustrate the ability of our sampling to handle discrete parameters. In both examples, one noise inputs used in the procedural model can be switched between several different types of noise. Out of the thousands of sampled solutions, we pick three that have different settings of the discrete parameter where the (log) pdf values decrease from sample-1 to sample-3.
• Figure 6: Results of our MCMC sampling on real inputs. For each example, the first column is the real target image (photo). We show MCMC samples in the other columns, where sample-1 and sample-2 are chosen closer to the peak of the posterior distribution, and sample-3 is further away. Note that the target images for Plaster-4 and Wood-5 are captured under natural illumination, while the corresponding synthetic images still assume collocated flash illumination; despite this mismatch, the estimated material parameters are still reasonable. Note, target images for Leather-4, Leather-6 and Wood-4 are from the publicly released dataset of Aittala2016. For more results please refer to supplemental materials.
• Figure 7: Comparison with mismatched forward models. With an inappropriate model as the prior, it would only match the global color but missing all the details.