Recasting Regional Lighting for Shadow Removal

TL;DR

This work tackles shadow removal by explicitly decoupling illumination and texture degradation. It introduces a two-stage framework: a shadow-aware decomposition network estimates reflectance $\mathbf{R}_{\text{s}}$ and illumination $\mathbf{L}_{\text{s}}$, followed by a bilateral correction network that first recasts local lighting via a diffusion-based Local Lighting Correction and then restores textures using an Illumination-Guided Texture Restoration conditioned on the corrected illumination. The method is supported by a Retinex-based regularization and self-supervised losses, and it includes manually annotated shadow masks for SRD to enable fair benchmarking. Experimental results on SRD, ISTD, and ISTD+ demonstrate state-of-the-art performance, with substantial improvements in shadow-region RMSE and favorable qualitative results, alongside detailed ablations and robustness analyses. The approach advances shadow removal by separating lighting from texture and guiding texture recovery with corrected illumination, offering practical improvements for downstream vision tasks in shadowed scenes.

Abstract

Removing shadows requires an understanding of both lighting conditions and object textures in a scene. Existing methods typically learn pixel-level color mappings between shadow and non-shadow images, in which the joint modeling of lighting and object textures is implicit and inadequate. We observe that in a shadow region, the degradation degree of object textures depends on the local illumination, while simply enhancing the local illumination cannot fully recover the attenuated textures. Based on this observation, we propose to condition the restoration of attenuated textures on the corrected local lighting in the shadow region. Specifically, We first design a shadow-aware decomposition network to estimate the illumination and reflectance layers of shadow regions explicitly. We then propose a novel bilateral correction network to recast the lighting of shadow regions in the illumination layer via a novel local lighting correction module, and to restore the textures conditioned on the corrected illumination layer via a novel illumination-guided texture restoration module. We further annotate pixel-wise shadow masks for the public SRD dataset, which originally contains only image pairs. Experiments on three benchmarks show that our method outperforms existing state-of-the-art shadow removal methods.

Figures (7)

• Figure 1: Comparison of shadow removal results. Existing methods (b-e) may fail to completely remove the shadow in the homogenous region and to recover the details in the textured region. Our method explicitly estimates the reflectance (f) and illumination (g) of the shadow image, based on which we recast the lighting and correct the texture in the shadow region, resulting in a more visually pleasing prediction (h).
• Figure 2: Method Overview. Given a shadow image $\mathbf{I}_{\text{s}}$ and a shadow mask $\mathbf{I}_{\text{m}}$ as input, the proposed method first decomposes the shadow image into a reflectance layer $\mathbf{R}_{\text{s}}$ and an illumination layer $\mathbf{L}_{\text{s}}$ via the shadow-aware decomposition network. $\mathbf{R}_{\text{s}}$, $\mathbf{L}_{\text{s}}$, and image features through skip-connections are then fed into the bilateral correction network for lighting correction via the Local Lighting Correction (LLC) module to generate the shadow-free lighting $\hat{\mathbf{L}}_{\text{s}}$, and texture restoration via the Illumination-Guided Texture Restoration (IGTR) module, and output the prediction $\mathbf{\hat{I}}$. In LLC, t is the time step, $\mathbf{x}_{0}$ is $\mathbf{L}_{\text{s}}$ during inference.
• Figure 3: Two examples of our shadow-aware decomposition and final prediction results in real-world samples.
• Figure 4: Overview of the proposed Illumination-Guided Texture Restoration (IGTR) module. It aims to correct the textures with the guidance of the recovered local lighting.
• Figure 5: Visual comparisons with state-of-the-art shadow removal methods on real-world samples.