Multi-illuminant Color Constancy via Multi-scale Illuminant Estimation and Fusion
Hang Luo, Rongwei Li, Jinxing Liang
TL;DR
This work tackles multi-illuminant color constancy by introducing a coarse-to-fine, multi-scale framework that estimates and fuses pixel-wise illuminant maps from three image scales. It represents the final illuminant distribution as a linear combination of multi-grained components estimated by three parallel IEM branches and fused through an Attentional Illuminant Fusion Module, with the fusion weights learned per pixel. The method leverages a variant of U-Net (IEM) for scale-specific illuminant maps and uses a linear fusion equation $I_{final} = I_{l} \times W_{l} + I_{m} \times W_{m} + I_{s} \times W_{s}$ guided by an attention mechanism, optimized via mean angular error. Experiments on the LSMI dataset demonstrate state-of-the-art performance and robust handling of scale variation, indicating strong practical impact for auto white balance and downstream vision tasks.
Abstract
Multi-illuminant color constancy methods aim to eliminate local color casts within an image through pixel-wise illuminant estimation. Existing methods mainly employ deep learning to establish a direct mapping between an image and its illumination map, which neglects the impact of image scales. To alleviate this problem, we represent an illuminant map as the linear combination of components estimated from multi-scale images. Furthermore, we propose a tri-branch convolution networks to estimate multi-grained illuminant distribution maps from multi-scale images. These multi-grained illuminant maps are merged adaptively with an attentional illuminant fusion module. Through comprehensive experimental analysis and evaluation, the results demonstrate the effectiveness of our method, and it has achieved state-of-the-art performance.