Neural Importance Sampling of Many Lights

TL;DR

This work tackles efficient direct illumination in scenes with many lights by learning a spatially varying light-selection distribution that guides Monte Carlo sampling. A neural network predicts cluster-level light probabilities and is trained online by minimizing $D_{KL}(q, p_\theta)$ to a target distribution derived from per-light contributions, enabling adaptive importance sampling without heavy spatial data structures. To scale to thousands of lights, the method combines neural predictions with a light hierarchy through a residual learning scheme that augments a baseline cluster distribution $w_c$ via $p_\theta(c) = \frac{e^{\left[ \log(w_c) + f_\theta(\mathbf{x}, \omega_o)[c] \right]}}{\sum_s e^{\left[ \log(w_s) + f_\theta(\mathbf{x}, \omega_o)[s] \right]}}$, accelerating convergence. Empirical results across diverse, complex scenes show superior fidelity and faster convergence than prior many-light techniques, with robust performance in challenging lighting and visibility conditions.

Abstract

We propose a neural approach for estimating spatially varying light selection distributions to improve importance sampling in Monte Carlo rendering, particularly for complex scenes with many light sources. Our method uses a neural network to predict the light selection distribution at each shading point based on local information, trained by minimizing the KL-divergence between the learned and target distributions in an online manner. To efficiently manage hundreds or thousands of lights, we integrate our neural approach with light hierarchy techniques, where the network predicts cluster-level distributions and existing methods sample lights within clusters. Additionally, we introduce a residual learning strategy that leverages initial distributions from existing techniques, accelerating convergence during training. Our method achieves superior performance across diverse and challenging scenes.

Figures (11)

• Figure 1: We illustrate a scene with three light sources of increasing brightness. Accurate light sampling requires considering factors such as light intensity, the BRDF, and visibility. The light with the highest contribution at each shading point, accounting for all factors, is indicated by red arrows. Sampling based solely on illumination magnitude leads to over-sampling light 2 at shading point $\mathbf{x}_1$, even though it does not contribute due to the directional constraints of the BRDF. Similarly, methods that consider both light intensity and the BRDF Estevez_2018_CGITYuksel_2019_HPG heavily sample light 1, which is occluded and thus does not contribute to the reflected light. Properly incorporating all three factors reveals that light 0 should be sampled more frequently at $\mathbf{x}_1$. While learning-based techniques Vevoda_2018_TOGPantaleoni_2019_arXivWang_2021_TOG incorporate visibility into their framework, they compute a single distribution for clusters of shading points (indicated by the gray dashed box), failing to capture critical local variations. For instance, light 0 dominates at $\mathbf{x}_1$, whereas light 2 contributes most at the neighboring point $\mathbf{x}_2$, underscoring the importance of spatially varying distributions.
• Figure 2: We present an overview of our technique. (left) The network uses local information at the shading point to estimate cluster probabilities in a residual manner. The baseline cluster distribution, $\mathbf{w}$, is precomputed using existing approaches and remains fixed throughout the process. (middle) A light is sampled in two stages: first, a cluster (one of the red nodes) is selected based on the estimated cluster distribution, and then the tree is stochastically traversed (green lines) to choose a specific light within the cluster. (right) Points are subsequently sampled within the selected light source, and paths are traced. These samples are used to train the network and refine the estimated cluster distribution. This entire process is repeated iteratively until the desired number of samples is traced. Since the network estimates the cluster PMF in a residual manner, it can guide sampling from the beginning of the rendering process.
• Figure 3: Given a light hierarchy constructed using an existing method Estevez_2018_CGITYuksel_2019_HPG, we select nodes at a specific level ($k = 3$ in this example) as light clusters, with the network estimating their probabilities $p_\theta(c)$. The probability of selecting an individual light (e.g., the blue leaf node) is computed as the product of the cluster probability and the conditional probability of the light given the cluster, $p(y|c)$. The conditional probability is obtained by multiplying the probabilities of the child nodes along the tree traversal path (green lines). Note that the grayed-out nodes are used solely for defining the global clusters and are not involved in sampling or evaluating the PMF.
• Figure 4: Equal-time comparison of ATS Estevez_2018_CGIT, SLCRT lin2020real, ReSTIR Bitterli_2020_TOG, VARL Wang_2021_TOG, and our method. The time budget increases with scene complexity and resolution. For Bathroom, Bedroom and Living Room, the budget is 5 seconds; Zero Day and San Miguel 10 seconds.
• Figure 5: Equal-sample comparison against several state-of-the-art methods. We use 128 spp for all the scenes.