Off the Planckian Locus: Using 2D Chromaticity to Improve In-Camera Color

TL;DR

This work challenges the conventional reliance on 1D CCT-based interpolation for in-camera color correction by introducing CST-MLP, a lightweight 2D chromaticity-based neural predictor that directly learns color space transforms. Training on a diverse lightbox dataset that includes non-Planckian LEDs, CST-MLP predicts a global 3×3 CST from 2D chromaticity inputs, preserving efficiency while improving color accuracy across single- and multi-illuminant scenes. Across extensive evaluations on lightbox, in-the-wild images, and the LSMI dataset, the 2D approach consistently outperforms traditional 1D methods and per-pixel MLPs, with substantial angular-error and ΔE2000 improvements and minimal computational overhead. The work also offers practical deployment avenues, including LUT conversion and extensions to synthesis, further solidifying its potential for industry adoption in modern LED-dominated lighting environments.

Abstract

Traditional in-camera colorimetric mapping relies on correlated color temperature (CCT)-based interpolation between pre-calibrated transforms optimized for Planckian illuminants such as CIE A and D65. However, modern lighting technologies such as LEDs can deviate substantially from the Planckian locus, exposing the limitations of relying on conventional one-dimensional CCT for illumination characterization. This paper demonstrates that transitioning from 1D CCT (on the Planckian locus) to a 2D chromaticity space (off the Planckian locus) improves colorimetric accuracy across various mapping approaches. In addition, we replace conventional CCT interpolation with a lightweight multi-layer perceptron (MLP) that leverages 2D chromaticity features for robust colorimetric mapping under non-Planckian illuminants. A lightbox-based calibration procedure incorporating representative LED sources is used to train our MLP. Validated across diverse LED lighting, our method reduces angular reproduction error by 22% on average in LED-lit scenes, maintains backward compatibility with traditional illuminants, accommodates multi-illuminant scenes, and supports real-time in-camera deployment with negligible additional computational cost.

Figures (20)

• Figure 1: 1D Planckian interpolation vs. our 2D chromaticity-based prediction. Traditional 1D methods perform adequately for light sources whose CCTs lie close to the Planckian locus but are notably less accurate for light sources (namely, LEDs) that lie off the Planckian locus. Our approach leverages 2D chromaticity coordinates for more robust color reproduction across all illuminant types. Heat maps show angular reproduction error, from $0^\circ$ (dark blue) to $15^\circ$ (dark red), demonstrating comparable performance along the Planckian curve but substantial improvements in LED-dominated regions where existing methods break down.
• Figure 2: Top: Camera pipeline overview. Early stages convert raw sensor responses to colorimetric values, while later stages transform colors to a photo-finished display-referred space Hakki_eccv16. This paper aims to achieve maximum accuracy in the colorimetric stage. Middle: Colorimetric mapping stage. When an image is captured under arbitrary illumination, the estimated illumination is used to remove color cast via a diagonal white-balance $3\!\times\!3$ matrix (i.e., $W^{l_a}_D$), then mapped to CIE XYZ with an illumination-specific $3\!\times\!3$ CST matrix (i.e., $T_{l_a}$). Bottom: 2D chromaticity CST prediction vs. 1D CCT interpolation. CST interpolation approaches (e.g., $T_{l_a} = gT_{2500 K} + (1-g)T_{6500K}$) assume CST parameters can be derived from a CCT-based interpolation, given calibrated illuminants along the Planckian Locus. CCT-based interpolation works poorly for non-Planckian illuminants. We propose a 2D CST-MLP that takes chromaticity coordinates as input and directly predicts the full $3\!\times\!3$ CST matrix.
• Figure 3:
• Figure 4: Colorimetric mapping results on laboratory dataset. Results for two non-Planckian LED sources: green LED (top half) and blue LED (bottom half) illuminants, captured using Canon and Pixel cameras. Each half starts with the illuminant's CCT and xy chromaticity and contains two rows: the top row shows the raw-RGB color chart and display-referred renderings; the bottom row shows the white-balanced color chart and error maps. Methods: 2-CST Hakki_CVPR18, 1D EXPINV expinv, 1D CST-MLP, 2D EXPINV expinv, 2D CST-MLP, and ground truth. Key findings: (1) Traditional 1D methods exhibit substantial errors for off-locus illuminants. (2) 2D chromaticity representation improves existing methods (compare 1D vs. 2D for EXPINV and CST-MLP).
• Figure 5: Computational cost comparison. We plot the MACs (log scale) at different resolutions for 2-CST AdobeDNG interpolation, EXPINV expinv, and 2D CST-MLP expinv. Our CST-MLP has nearly the same computational cost as traditional methods, since the CST is computed once and applied to the entire image. In contrast, approaches such as EXPINV must evaluate the MLP at every pixel.