MSSIDD: A Benchmark for Multi-Sensor Denoising

TL;DR

This paper introduces a new benchmark, the Multi-Sensor SIDD (MSSIDD) dataset, which is the first raw-domain dataset designed to evaluate the sensor transferability of denoising models, and proposes a sensor consistency training framework that enables denoising models to learn the sensor-invariant features, thereby facilitating the generalization of the consistent model to unseen sensors.

Abstract

The cameras equipped on mobile terminals employ different sensors in different photograph modes, and the transferability of raw domain denoising models between these sensors is significant but remains sufficient exploration. Industrial solutions either develop distinct training strategies and models for different sensors or ignore the differences between sensors and simply extend existing models to new sensors, which leads to tedious training or unsatisfactory performance. In this paper, we introduce a new benchmark, the Multi-Sensor SIDD (MSSIDD) dataset, which is the first raw-domain dataset designed to evaluate the sensor transferability of denoising models. The MSSIDD dataset consists of 60,000 raw images of six distinct sensors, derived through the degeneration of sRGB images via different camera sensor parameters. Furthermore, we propose a sensor consistency training framework that enables denoising models to learn the sensor-invariant features, thereby facilitating the generalization of the consistent model to unseen sensors. We evaluate previous arts on the newly proposed MSSIDD dataset, and the experimental results validate the effectiveness of our proposed method. Our dataset is available at https://www.kaggle.com/datasets/sjtuwh/mssidd.

Figures (7)

• Figure 1: The inverse transformation of ISP pipeline to obtain degenerated raw images.
• Figure 2: The impact of white balance and color correction of different sensors on standard Macbeth color-checker.
• Figure 3: Noisy and clean image pairs of six sensors. Demosaicing is applied for better visualization.
• Figure 4: Analysis of noise statistic of six sensors. We display the SNR and total noise variance concerning ADU. We also present the relationship between read and shot noise and camera ISO.
• Figure 5: Model Structure. Intra- and inter-image sensor consistency supervision and adversarial learning are equipped to facilitate sensor generalization.