PhaSR: Generalized Image Shadow Removal with Physically Aligned Priors

TL;DR

Shadow removal under diverse lighting requires disentangling illumination from reflectance. PhaSR tackles this with dual-level physically aligned priors: PAN provides global illumination normalization via Gray-world correction and log-domain Retinex, while GSRA harmonizes depth-based geometric priors with DINO-V2 semantic priors through cross-modal differential attention, expressed as $\mathbf{A}_{\text{rect}} = \mathbf{A}_{\text{sem}} - \lambda \cdot \mathbf{A}_{\text{geo}}$. The approach achieves state-of-the-art results on challenging ambient-light benchmarks (e.g., Ambient6K) and competitive performance on standard shadow datasets, with low FLOPs ($55.63$ GFLOPs) and fast runtime ($87.9$ ms on $640\times480$ images). By integrating closed-form normalization and explicit prior alignment, PhaSR generalizes from single-light direct shadows to multi-source ambient illumination, enabling robust shadow-free restoration in real-world scenes; code is available at GitHub.

Abstract

Shadow removal under diverse lighting conditions requires disentangling illumination from intrinsic reflectance, a challenge compounded when physical priors are not properly aligned. We propose PhaSR (Physically Aligned Shadow Removal), addressing this through dual-level prior alignment to enable robust performance from single-light shadows to multi-source ambient lighting. First, Physically Aligned Normalization (PAN) performs closed-form illumination correction via Gray-world normalization, log-domain Retinex decomposition, and dynamic range recombination, suppressing chromatic bias. Second, Geometric-Semantic Rectification Attention (GSRA) extends differential attention to cross-modal alignment, harmonizing depth-derived geometry with DINO-v2 semantic embeddings to resolve modal conflicts under varying illumination. Experiments show competitive performance in shadow removal with lower complexity and generalization to ambient lighting where traditional methods fail under multi-source illumination. Our source code is available at https://github.com/ming053l/PhaSR.

Figures (16)

• Figure 1: Results on indoor-synthesized dataset omnisr. Compared with OmniSR omnisr and DenseSR densesr, PhaSR with the proposed GSRA achieves more accurate boundary localization and recovers fine reflectance details even under complex indirect illumination.
• Figure 2: Intermediate feature visualization. Existing methods struggle to leverage physical priors without valid shadow masks under complex environmental lighting. In contrast, our PhaSR precisely highlights and restores shadow regions in both bottleneck and decoder stages, demonstrating strong generalization.
• Figure 3: Overview of PhaSR: Physically Aligned Shadow Removal. PhaSR achieves physical alignment through two synergistic stages. Stage 1 (Sec. \ref{['sec:pan']}): PAN performs model-free illumination normalization via Gray-world color correction, log-domain Retinex decomposition ($\log \mathbf{I} = \log \mathbf{R} + \log \mathbf{S}$), and dynamic range recombination, suppressing chromatic bias while preserving reflectance cues. Stage 2 (Sec. \ref{['sec:gsra']}): The multi-scale Transformer encoder-decoder integrates explicit physical priors—frozen DINO-v2 dinov2 semantic embeddings at encoder stages and DepthAnything-v2 depthganythingv2 geometric priors (depth, normals) at the bottleneck—aligned through GSRA's cross-modal differential attention ($\mathbf{A}_{\text{rect}} = \mathbf{A}_{\text{sem}} - \lambda \cdot \mathbf{A}_{\text{geo}}$). This dual-stage physical alignment—global illumination correction followed by local geometric-semantic rectification—enables robust reflectance recovery under complex lighting without requiring shadow masks.
• Figure 4: Overview of the proposed PAN. It performs model-free illumination correction through three stages: (1) Global color normalization removes chromatic bias, (2) log-domain Retinex decomposition separates reflectance $\hat{\mathbf{R}}$ from illumination $\hat{\mathbf{S}}$ via closed-form operations, and (3) recombination produces the illumination-consistent output $\hat{\mathbf{I}}$.
• Figure 5: Overview of the proposed GSRA. Semantic and geometric features are projected into modality-specific key–value spaces and queried with shared tokens. Rectification aligns the two modalities through soft attention balancing, resolving illumination inconsistencies while preserving geometric stability.