Neural Product Importance Sampling via Warp Composition

TL;DR

This work tackles the long-standing challenge of efficiently sampling the product of illumination and material terms in Monte Carlo rendering. It introduces a novel warp-composition framework that splits the learning task into a small, conditioned neural spline head warp and a large, unconditional tail warp derived from the environment map, enabling a near-exact product sampling distribution while keeping inference fast. Training optimizes a forward KL objective with entropic regularization to robustly fit the target product density, and the tail warp is discretized for fast lookups, with a baked variant offering practical speedups. Across cosine-weighted emitter sampling, microfacet BRDFs, neural materials, and shadow-catcher compositing, the method achieves significant variance reductions over MIS at equal time and sample counts, validating the approach's practicality and effectiveness in real rendering pipelines.

Abstract

Achieving high efficiency in modern photorealistic rendering hinges on using Monte Carlo sampling distributions that closely approximate the illumination integral estimated for every pixel. Samples are typically generated from a set of simple distributions, each targeting a different factor in the integrand, which are combined via multiple importance sampling. The resulting mixture distribution can be far from the actual product of all factors, leading to sub-optimal variance even for direct-illumination estimation. We present a learning-based method that uses normalizing flows to efficiently importance sample illumination product integrals, e.g., the product of environment lighting and material terms. Our sampler composes a flow head warp with an emitter tail warp. The small conditional head warp is represented by a neural spline flow, while the large unconditional tail is discretized per environment map and its evaluation is instant. If the conditioning is low-dimensional, the head warp can be also discretized to achieve even better performance. We demonstrate variance reduction over prior methods on a range of applications comprising complex geometry, materials and illumination.

Figures (11)

• Figure 1: Naively fitting a normalizing flow (NF) model to the product of a complex unconditioned density $p_1$ (image) and a simple conditioned density $p_2$ (Gaussian with parameterized mean $\mu$) yields a poor result. The model is tasked with simultaneously learning the intricate shape of $p_1$and the variations in $\mu$. Instead, we apply a $p_1$ warp to the NF-model output, which drastically simplifies the shape of the distribution it needs to learn. The result is a near-perfect fit with an equal number of NF parameters.
• Figure 2: Given a shading condition, our model maps uniform points through two warps to produce samples distributed approximately proportionally to a target product density. The shape of our intermediate density is coarse, similarly to a naive product fit (see \ref{['fig:2d-example']}), but leads to a precise fit when mapped through the tail warp.
• Figure 3: Given a shading condition $\mathbf{c}$ (view direction $\bm{\omega}_\mathrm{o}$, surface normal $\mathbf{n}$ and material descriptor $\bm{\rho}$), a conditioner encoder first produces a latent vector $\bm{\xi}$. The vectors $\mathbf{c}$ and $\bm{\xi}$ condition two coupling layers, each warping samples via a circular piecewise rational quadratic (RQ) spline whose parameters (i.e., knot positions and derivatives) are inferred by a spline network. The output $y = (y_0,y_1)$ is then passed through our tail warp to produce the final sample $x$ which is converted to a direction $\bm{\omega}$ via lat-long mapping.
• Figure 4: Using a hierarchical tail warp exhibits discontinuities and hinders head-warp optimization. We do not show the standard marginal row-column warp as it performs even worse. In contrast, our NF-based tail warp makes for a smooth mapping and halves the MSE of images rendered with our full model. Colors encode the mapping from the unit square.
• Figure 5: Two intermediate distributions (at scene points A & B) learned by our head-warp NF model (top plots) and their low-resolution bakes (bottom plots) used for fast inference on the Diorama scene. Our baked scheme roughly matches the run time of the MIS baseline and achieves a $2.4\times$ MSE reduction over it, with minimal noise increase over the slower NF model. Note that most of the remaining noise for our method comes from visibility.