Towards Image Ambient Lighting Normalization

TL;DR

The large-scale high-resolution dataset Ambient6K is introduced, comprising samples obtained from multiple light sources and including self-shadows resulting from complex geometries, which is the first of its kind, and IFBlend, a novel strong baseline that maximizes Image-Frequency joint entropy to selectively restore local areas under different lighting conditions, without relying on shadow localization priors.

Abstract

Lighting normalization is a crucial but underexplored restoration task with broad applications. However, existing works often simplify this task within the context of shadow removal, limiting the light sources to one and oversimplifying the scene, thus excluding complex self-shadows and restricting surface classes to smooth ones. Although promising, such simplifications hinder generalizability to more realistic settings encountered in daily use. In this paper, we propose a new challenging task termed Ambient Lighting Normalization (ALN), which enables the study of interactions between shadows, unifying image restoration and shadow removal in a broader context. To address the lack of appropriate datasets for ALN, we introduce the large-scale high-resolution dataset Ambient6K, comprising samples obtained from multiple light sources and including self-shadows resulting from complex geometries, which is the first of its kind. For benchmarking, we select various mainstream methods and rigorously evaluate them on Ambient6K. Additionally, we propose IFBlend, a novel strong baseline that maximizes Image-Frequency joint entropy to selectively restore local areas under different lighting conditions, without relying on shadow localization priors. Experiments show that IFBlend achieves SOTA scores on Ambient6K and exhibits competitive performance on conventional shadow removal benchmarks compared to shadow-specific models with mask priors. The dataset, benchmark, and code are available at https://github.com/fvasluianu97/IFBlend.

Figures (9)

• Figure 1: Motivation. (a) Conventional restoration tasks, such as deraining, typically aim to restore information across the entire image, evident from the error map. However, in the context of lighting normalization, which has been traditionally oversimplified as shadow removal with only one lighting source and smooth surface, the objective becomes the recovery of local information within the shadow mask. Nonetheless, this simplified setup lacks realism for everyday scenarios. To address this limitation, we propose the Ambient Lighting Normalization (ALN) task, which explores the complexities of shadow interactions. Model-wise, we introduce a robust baseline by capitalizing on the synergy between image and frequency domains. Unlike conventional approaches that adopt a dual-branch fusion design, as depicted in (b), we propose a shrinkage multi-band fusion technique (c) to maximize their joint entropy.
• Figure 2: Visualization of shadow configurations. A: Self-casted shadow configuration with a unique directional light. Note the non-linear intensity in the shadow's penumbra towards the edges. B: A second directional light is added to the setup. Note the different shadow intensities in different areas, depending on the intensity of each light source.
• Figure 3: Left: The professional setup used for data acquisition. The directional lights $L_1 - L3$ are used to cast shadows in the scene, while the softbox lights counter the self-shadows casted by rough surfaces. Right: The typical lighting scenario in ISTD ISTDwang2018STCGAN (top left), WSRD Vasluianu_2023_WSRD (top right), and Ambient6K (bottom). Note the non-uniform lighting distribution due to caustics (A) or rough surfaces (B), and the increased complexity scenes represented at data level.
• Figure 4: Top: Input samples from the Ambient6K benchmark. Note the high complexity of the scenes, with various contents and textures. Combined with the complex illumination model, the shadow patterns are characterized by increased variety in terms of shape, intensity and interactions. Bottom: Equivalent ground-truth images acquired in optimal lightning conditions.
• Figure 5: (a) The architecture of our proposed IFBlend for ALN involves splitting the image feature into low- and high-frequency domains, treating them independently. (b) The encoder-decoder pair's inner structure aims to progressively compute, refine, and combine features across various frequency domains. The final output undergoes a shrinkage fusion process to achieve enhanced lighting normalization. The Global Context Branch (GCB) is based on ConvNext liu2022convnet.