Adaptive Domain Learning for Cross-domain Image Denoising

TL;DR

The proposed ADL training scheme automatically removes the data in the source domain that are harmful to fine-tuning a model for the target domain (some data are harmful as adding them during training lowers the performance due to domain gaps).

Abstract

Different camera sensors have different noise patterns, and thus an image denoising model trained on one sensor often does not generalize well to a different sensor. One plausible solution is to collect a large dataset for each sensor for training or fine-tuning, which is inevitably time-consuming. To address this cross-domain challenge, we present a novel adaptive domain learning (ADL) scheme for cross-domain RAW image denoising by utilizing existing data from different sensors (source domain) plus a small amount of data from the new sensor (target domain). The ADL training scheme automatically removes the data in the source domain that are harmful to fine-tuning a model for the target domain (some data are harmful as adding them during training lowers the performance due to domain gaps). Also, we introduce a modulation module to adopt sensor-specific information (sensor type and ISO) to understand input data for image denoising. We conduct extensive experiments on public datasets with various smartphone and DSLR cameras, which show our proposed model outperforms prior work on cross-domain image denoising, given a small amount of image data from the target domain sensor.

Figures (3)

• Figure 1: The overall pipeline of our adaptive domain learning (ADL) algorithm. The network parameter $\theta_{0}$ is first initialized, then the small target domain training set will be used to train a model with parameter $\theta$. In the source domain adaptive learning stage, in iteration $t$, data from the source domain will be used to update the network parameter from $\theta_{t-1}$ to $\theta'$. Then a dynamic validation set will judge whether the data is useful. If so, set $\theta_{t} = \theta'$ and repeat the process. If not, retrieve the network parameter from $\theta'$ to $\theta_{t-1}$. Finally, the target domain data will be used to fine-tune $\theta'$ to $\theta_{T}$
• Figure 2: The error map of our method compares against state-of-the-art approaches. The first row is the result from the SIDD dataset, and the second row is the result from the SID dataset. We can see that our method is able to generate the image with smaller errors and less noise compared to previous work.
• Figure 3: The ablation study of the size of the validation set. Our dynamic validation set strategy can overcome the overfitting problem when the size of the target domain dataset is extremely small.