Fast Local Neural Regression for Low-Cost, Path Traced Lambertian Global Illumination

TL;DR

This work proposes incorporating a neural network into a computationally-efficient local linear model-based denoiser, and demonstrates faithful single-frame reconstruction of global illumination for Lambertian scenes at very low sample counts and for low computational cost.

Abstract

Despite recent advances in hardware acceleration of ray tracing, real-time ray budgets remain stubbornly limited at a handful of samples per pixel (spp) on commodity hardware, placing the onus on denoising algorithms to achieve high visual quality for path traced global illumination. Neural network-based solutions give excellent result quality at the cost of increased execution time relative to hand-engineered methods, making them less suitable for deployment on resource-constrained systems. We therefore propose incorporating a neural network into a computationally-efficient local linear model-based denoiser, and demonstrate faithful single-frame reconstruction of global illumination for Lambertian scenes at very low sample counts (1spp) and for low computational cost. Other contributions include improving the quality and performance of local linear model-based denoising through a simplified mathematical treatment, and demonstration of the surprising usefulness of ambient occlusion as a guide channel. We also show how our technique is straightforwardly extensible to joint denoising and upsampling of path traced renders with reference to low-cost, rasterized guide channels.

Figures (9)

• Figure 1: In Fast Local Neural Regression, the guide image and indirect light are passed into a neural network, which extracts structural information for the linear model to exploit. Our Fast Local Regression algorithm efficiently fits noise-free guide channels (e.g. rasterised at low cost) to a noisy, path traced input image. Optimisations include fitting the model $\textbf{A}_k$ to 8x8 blocks, and use of cheap downsampling and separable blurs to collect local moments.
• Figure 2: A basic, differentiable implementation of the core local linear algorithm in PyTorch, with basic regularisation. It returns a $\left(3 \times H \times W\right)$ denoised RGB image.
• Figure 3: Cheaply rasterising two sets of corresponding guide channels, one at low resolution and one at high resolution, allows indirect lighting to be path traced at low resolution and jointly denoised and upsampled with minimal loss in quality.
• Figure 4: Effects of introducing aspects of the proposed method. FLR with surface normal and depth guides correctly reconstructs attached shadows, but typically misses cast shadows. These are recovered with the introduction of ambient occlusion as a guide channel, and neural network generated enhanced guides. The neural network also consumes the noisy input.
• Figure 5: Effects of using different combinations of guide channels for the network input in FLNR. All the examples include the noisy input radiance as an additional input. Although input radiance and surface normals are sufficient to get a denoised estimate of the frame, the filter looses a great amount of the structural information of the scene. Introducing depth and, in particular, ambient occlusion helps the filter restoring the lost structural information, while also improving the quality of denoised shadows.