Nonlinear Noise2Noise for Efficient Monte Carlo Denoiser Training

TL;DR

The paper tackles the challenge of training denoisers without clean targets in high-dynamic-range Monte Carlo rendering by extending Noise2Noise to nonlinear target processing. It provides a Jensen-gap–based framework to analyze and bound bias from nonlinearities, and demonstrates that carefully paired loss functions and tone-mapping can yield robust training with noisy data. The authors implement and test nonlinear Noise2Noise on a Monte Carlo denoiser, achieving HDDR denoising performance close to clean-reference baselines while using significantly noisier training data, particularly at low samples per pixel. This approach offers a practical path to efficient, weakly supervised denoising in challenging rendering pipelines and potentially beyond.

Abstract

The Noise2Noise method allows for training machine learning-based denoisers with pairs of input and target images where both the input and target can be noisy. This removes the need for training with clean target images, which can be difficult to obtain. However, Noise2Noise training has a major limitation: nonlinear functions applied to the noisy targets will skew the results. This bias occurs because the nonlinearity makes the expected value of the noisy targets different from the clean target image. Since nonlinear functions are common in image processing, avoiding them limits the types of preprocessing that can be performed on the noisy targets. Our main insight is that certain nonlinear functions can be applied to the noisy targets without adding significant bias to the results. We develop a theoretical framework for analyzing the effects of these nonlinearities, and describe a class of nonlinear functions with minimal bias. We demonstrate our method on the denoising of high dynamic range (HDR) images produced by Monte Carlo rendering. Noise2Noise training can have trouble with HDR images, where the training process is overwhelmed by outliers and performs poorly. We consider a commonly used method of addressing these training issues: applying a nonlinear tone mapping function to the model output and target images to reduce their dynamic range. This method was previously thought to be incompatible with Noise2Noise training because of the nonlinearities involved. We show that certain combinations of loss functions and tone mapping functions can reduce the effect of outliers while introducing minimal bias. We apply our method to an existing machine learning-based Monte Carlo denoiser, where the original implementation was trained with high-sample count reference images. Our results approach those of the original implementation, but are produced using only noisy training data.

Figures (6)

• Figure 1: (a) Tone mapping functions used for our experimental results. See Section \ref{['ssec:model_selection']} for details. (b) $J(y)$ functions for the listed $\varphi(\hat{y})$ functions resulting from all combinations of our studied loss functions and tone mapping functions. $J_-(y)$ and $J_+(y)$ functions are represented together as $\max(\lvert J_-(y)\rvert\,,\lvert J_+(y)\rvert)$, with $\epsilon=0.01$. See Section \ref{['ssec:jensen_bound']} for details on $J(y)$ functions, and Section \ref{['ssec:jensen_gap']} for a list of the $J(y)$ functions plotted above. (c) Graphical representation of the Bhatia-Davis inequality for bounded probability distributions with minimum $m=0$ and maximum $M$. See Section \ref{['ssec:minimizing']} for details.
• Figure 2: (a) Training losses, (b) Gradient norms, and (c) Validation rmse for all combinations of loss functions, tone mapping functions, and tone mapping placement. For the training losses, the units differ for the different loss functions, so comparisons in terms of loss value are not meaningful, and instead we are interested in the numerical stability of the loss landscapes. The training losses and gradient norms are smoothed with a centered median filter with a width of 2,000 iterations. The gradient norms were clipped at a value of $10^3$ during training, and so will not exceed this value. Training runs that end before 40,000 iterations were terminated due to numerical errors.
• Figure 3: Model outputs for a single validation image for all combinations of loss functions, tone mapping functions, and tone mapping placement
• Figure 4: (a) rmse and (b) dssim error metrics for each denoiser at various input spp, averaged over the sbmc test set
• Figure 5: Model outputs for a single validation image for all combinations of loss functions, tone mapping functions, and tone mapping placement