Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RL-AWB: Deep Reinforcement Learning for Auto White Balance Correction in Low-Light Night-time Scenes

Yuan-Kang Lee, Kuan-Lin Chen, Chia-Che Chang, Yu-Lun Liu

TL;DR

This work tackles the problem of automatic white balance in nighttime scenes, where low light and sensor noise destabilize traditional estimators. It introduces RL-AWB, a hybrid approach that couples a novel nighttime color-constancy algorithm, SGP-LRD, with a reinforcement-learning agent (SAC) to adapt two key parameters per image, achieving data-efficient learning and strong cross-sensor generalization. The LEVI dataset is presented as a multi-camera nighttime benchmark to evaluate cross-sensor robustness. Experimental results demonstrate that RL-AWB delivers competitive performance on nighttime data and generalizes well to daytime scenarios, highlighting the practical impact of combining statistically-grounded estimation with curriculum-guided RL for ISP parameter tuning.

Abstract

Nighttime color constancy remains a challenging problem in computational photography due to low-light noise and complex illumination conditions. We present RL-AWB, a novel framework combining statistical methods with deep reinforcement learning for nighttime white balance. Our method begins with a statistical algorithm tailored for nighttime scenes, integrating salient gray pixel detection with novel illumination estimation. Building on this foundation, we develop the first deep reinforcement learning approach for color constancy that leverages the statistical algorithm as its core, mimicking professional AWB tuning experts by dynamically optimizing parameters for each image. To facilitate cross-sensor evaluation, we introduce the first multi-sensor nighttime dataset. Experiment results demonstrate that our method achieves superior generalization capability across low-light and well-illuminated images. Project page: https://ntuneillee.github.io/research/rl-awb/

RL-AWB: Deep Reinforcement Learning for Auto White Balance Correction in Low-Light Night-time Scenes

TL;DR

This work tackles the problem of automatic white balance in nighttime scenes, where low light and sensor noise destabilize traditional estimators. It introduces RL-AWB, a hybrid approach that couples a novel nighttime color-constancy algorithm, SGP-LRD, with a reinforcement-learning agent (SAC) to adapt two key parameters per image, achieving data-efficient learning and strong cross-sensor generalization. The LEVI dataset is presented as a multi-camera nighttime benchmark to evaluate cross-sensor robustness. Experimental results demonstrate that RL-AWB delivers competitive performance on nighttime data and generalizes well to daytime scenarios, highlighting the practical impact of combining statistically-grounded estimation with curriculum-guided RL for ISP parameter tuning.

Abstract

Nighttime color constancy remains a challenging problem in computational photography due to low-light noise and complex illumination conditions. We present RL-AWB, a novel framework combining statistical methods with deep reinforcement learning for nighttime white balance. Our method begins with a statistical algorithm tailored for nighttime scenes, integrating salient gray pixel detection with novel illumination estimation. Building on this foundation, we develop the first deep reinforcement learning approach for color constancy that leverages the statistical algorithm as its core, mimicking professional AWB tuning experts by dynamically optimizing parameters for each image. To facilitate cross-sensor evaluation, we introduce the first multi-sensor nighttime dataset. Experiment results demonstrate that our method achieves superior generalization capability across low-light and well-illuminated images. Project page: https://ntuneillee.github.io/research/rl-awb/
Paper Structure (54 sections, 22 equations, 7 figures, 8 tables)

This paper contains 54 sections, 22 equations, 7 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Statistical Color Constancy.
  4. Learning-Based Color Constancy.
  5. Nighttime and Low-Light Color Constancy.
  6. Reinforcement Learning for ISP.
  7. Cross-Sensor Generalization.
  8. Hybrid Statistical-Learning Approaches.
  9. Curriculum Learning in RL.
  10. Method
  11. Nighttime Color Constancy Algorithm
  12. Salient Gray Pixel Detection.
  13. Local Variance Filtering.
  14. Color Deviation Filtering.
  15. Gray-pixel Confidence Weighting.
  16. ...and 39 more sections

Figures (7)

  • Figure 1: Overview of the proposed RL-AWB framework. (A) Given an input image, the proposed nighttime color constancy algorithm SGP-LRD estimates the scene illuminant conditioned on two hyperparameters (gray-pixel sampling percentage $N$ and Minkowski order $p$). (B) A SAC agent selects parameter updates based on image statistics and current AWB settings. (C) The policy outputs one action per parameter; actions are sampled, squashed by $\tanh$ to $[-1,1]$, and rescaled to valid ranges. (D) The rescaled actions update the two hyperparameters and are applied to SGP-LRD to produce the illuminant estimate. Repeat until the termination criterion is met.
  • Figure 2: Sample images from the proposed LEVI dataset with their corresponding Color Checker mask annotations. The dataset captures diverse nighttime scenes with complex mixed lighting, low illumination, and high ISO conditions.
  • Figure 3: Illuminant distribution over all the collected nighttime images in the LEVI and NCC datasets.
  • Figure 4: Normalized mean luminance histogram over all the collected nighttime images in the LEVI and NCC datasets.
  • Figure 5: Qualitative comparison of cross-dataset performance. Images are gamma-corrected for visualization. Top: train on LEVI, test on NCC; bottom: train on NCC, test on LEVI. Learning-based methods degrade under cross-dataset shift, while RL-AWB remains stable.
  • ...and 2 more figures