Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Learnable Color Correction Matrix for RAW Reconstruction

Anqi Liu, Shiyi Mu, Shugong Xu

TL;DR

Experimental results demonstrate that simulated RAW (simRAW) images generated by the proposed method provide performance improvements equivalent to those produced by more complex inverse ISP methods when pretraining RAW-domain object detectors, which highlights the effectiveness and practicality of the approach.

Abstract

Autonomous driving algorithms usually employ sRGB images as model input due to their compatibility with the human visual system. However, visually pleasing sRGB images are possibly sub-optimal for downstream tasks when compared to RAW images. The availability of RAW images is constrained by the difficulties in collecting real-world driving data and the associated challenges of annotation. To address this limitation and support research in RAW-domain driving perception, we design a novel and ultra-lightweight RAW reconstruction method. The proposed model introduces a learnable color correction matrix (CCM), which uses only a single convolutional layer to approximate the complex inverse image signal processor (ISP). Experimental results demonstrate that simulated RAW (simRAW) images generated by our method provide performance improvements equivalent to those produced by more complex inverse ISP methods when pretraining RAW-domain object detectors, which highlights the effectiveness and practicality of our approach.

A Learnable Color Correction Matrix for RAW Reconstruction

TL;DR

Experimental results demonstrate that simulated RAW (simRAW) images generated by the proposed method provide performance improvements equivalent to those produced by more complex inverse ISP methods when pretraining RAW-domain object detectors, which highlights the effectiveness and practicality of the approach.

Abstract

Autonomous driving algorithms usually employ sRGB images as model input due to their compatibility with the human visual system. However, visually pleasing sRGB images are possibly sub-optimal for downstream tasks when compared to RAW images. The availability of RAW images is constrained by the difficulties in collecting real-world driving data and the associated challenges of annotation. To address this limitation and support research in RAW-domain driving perception, we design a novel and ultra-lightweight RAW reconstruction method. The proposed model introduces a learnable color correction matrix (CCM), which uses only a single convolutional layer to approximate the complex inverse image signal processor (ISP). Experimental results demonstrate that simulated RAW (simRAW) images generated by our method provide performance improvements equivalent to those produced by more complex inverse ISP methods when pretraining RAW-domain object detectors, which highlights the effectiveness and practicality of our approach.
Paper Structure (11 sections, 7 equations, 4 figures, 3 tables)

This paper contains 11 sections, 7 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Method
  4. Unpaired RAW Reconstruction Teacher Model
  5. Learnable Color Correction Matrix
  6. Experiments
  7. Experiment Settings
  8. Main Results of RAW Reconstruction
  9. Main Results of RAW-domain Object Detection
  10. Ablation Studies
  11. Conclusion

Figures (4)

  • Figure 1: Overview of our proposed framework. The Unpaired-CycleR2R li2024efficient model serves as the teacher model, while our proposed LCCM functions as the student model. LCCM is composed of a single convolutional layer designed to imitate the learnable color correction matrix. The simRAW-T sRGB refers to RGB images generated offline using the teacher model. During training, pairs of sRGB images from the BDD100K dataset yu2020bdd100k and simRAW-T sRGB images are used to train LCCM. The simRAW-S sRGB refers to RGB images generated by the LCCM, and simRAW-S represents Bayer pattern RAW images, similar to simRAW-T.
  • Figure 2: Qualitative comparison of RAW reconstruction results for the asi 294mcpro camera style.
  • Figure 3: Histograms for RGB channel of sRGB images from the BDD100K dataset yu2020bdd100k, simRAW-S sRGB and simRAW-T sRGB images for the huawei P30pro camera style. Frequency denotes the occurrence rate of each pixel. Intensity denotes the brightness level of color.
  • Figure 4: Ablation study on the number of training samples for the iphone XSmax camera style.