Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Pixel-Wise Color Constancy via Smoothness Techniques in Multi-Illuminant Scenes

Umut Cem Entok, Firas Laakom, Farhad Pakdaman, Moncef Gabbouj

TL;DR

This work tackles color constancy under scenes with multiple, spatially varying illuminants by predicting per-pixel illumination maps. It combines a U-Net-based estimator operating in the log-chrominance space with Total Variation loss to impose spatial smoothness, and uses a bilateral filter for photorealistic post-processing. To handle noisy ground-truth illumination, the authors apply label smoothing to the per-pixel weights. On the Large Scale Multi-Illuminant (LSMI) dataset, their PWCC methods achieve state-of-the-art recovery angular errors, with a mean around $2^\circ$, and show notable improvements on difficult cases; the approach offers practical improvements for robust white balancing in real-world, multi-illuminant scenes.

Abstract

Most scenes are illuminated by several light sources, where the traditional assumption of uniform illumination is invalid. This issue is ignored in most color constancy methods, primarily due to the complex spatial impact of multiple light sources on the image. Moreover, most existing multi-illuminant methods fail to preserve the smooth change of illumination, which stems from spatial dependencies in natural images. Motivated by this, we propose a novel multi-illuminant color constancy method, by learning pixel-wise illumination maps caused by multiple light sources. The proposed method enforces smoothness within neighboring pixels, by regularizing the training with the total variation loss. Moreover, a bilateral filter is provisioned further to enhance the natural appearance of the estimated images, while preserving the edges. Additionally, we propose a label-smoothing technique that enables the model to generalize well despite the uncertainties in ground truth. Quantitative and qualitative experiments demonstrate that the proposed method outperforms the state-of-the-art.

Pixel-Wise Color Constancy via Smoothness Techniques in Multi-Illuminant Scenes

TL;DR

This work tackles color constancy under scenes with multiple, spatially varying illuminants by predicting per-pixel illumination maps. It combines a U-Net-based estimator operating in the log-chrominance space with Total Variation loss to impose spatial smoothness, and uses a bilateral filter for photorealistic post-processing. To handle noisy ground-truth illumination, the authors apply label smoothing to the per-pixel weights. On the Large Scale Multi-Illuminant (LSMI) dataset, their PWCC methods achieve state-of-the-art recovery angular errors, with a mean around , and show notable improvements on difficult cases; the approach offers practical improvements for robust white balancing in real-world, multi-illuminant scenes.

Abstract

Most scenes are illuminated by several light sources, where the traditional assumption of uniform illumination is invalid. This issue is ignored in most color constancy methods, primarily due to the complex spatial impact of multiple light sources on the image. Moreover, most existing multi-illuminant methods fail to preserve the smooth change of illumination, which stems from spatial dependencies in natural images. Motivated by this, we propose a novel multi-illuminant color constancy method, by learning pixel-wise illumination maps caused by multiple light sources. The proposed method enforces smoothness within neighboring pixels, by regularizing the training with the total variation loss. Moreover, a bilateral filter is provisioned further to enhance the natural appearance of the estimated images, while preserving the edges. Additionally, we propose a label-smoothing technique that enables the model to generalize well despite the uncertainties in ground truth. Quantitative and qualitative experiments demonstrate that the proposed method outperforms the state-of-the-art.
Paper Structure (14 sections, 14 equations, 3 figures, 2 tables)

This paper contains 14 sections, 14 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed Method
  4. Method Overview
  5. Ground Truth Illumination Model
  6. Color Space Transformation
  7. Estimation Model
  8. Label Smoothing
  9. Total Variation Loss
  10. Bilateral Filtering
  11. Experiments and Discussion
  12. Evaluation Metrics
  13. Conclusion
  14. ACKNOWLEDGEMENT

Figures (3)

  • Figure 1: Overview of the proposed method: The U-Net multi-illuminant estimation model processes input images and smoothed ground truth images within the log-chrominance color space. During training, the model learns an illumination mapping between input and ground truth images by minimizing Total Variation Loss and L2 loss. The Bilateral Filter is then applied to the estimated images. The final output is the white-balanced image.
  • Figure 2: Distributions of $\alpha_{raw}(x,y)$ and $\alpha_{smooth}$ parameters
  • Figure 3: Visualization results on different two-illuminant images of LSMI Dataset kim2021large. From left to right, input image, baseline (LSMI-U-Net kim2021large), PWCC_v1, PWCC_v2 predictions, and the ground truth image, respectively.