FASR-Net: Unsupervised Shadow Removal Leveraging Inherent Frequency Priors

TL;DR

A novel unsupervised Frequency Aware Shadow Removal Network (FASR-Net) is proposed, which leverages the inherent frequency characteristics of shadow regions to enhance shadow details within specific frequency bands.

Abstract

Shadow removal is challenging due to the complex interaction of geometry, lighting, and environmental factors. Existing unsupervised methods often overlook shadow-specific priors, leading to incomplete shadow recovery. To address this issue, we propose a novel unsupervised Frequency Aware Shadow Removal Network (FASR-Net), which leverages the inherent frequency characteristics of shadow regions. Specifically, the proposed Wavelet Attention Downsampling Module (WADM) integrates wavelet-based image decomposition and deformable attention, effectively breaking down the image into frequency components to enhance shadow details within specific frequency bands. We also introduce several new loss functions for precise shadow-free image reproduction: a frequency loss to capture image component details, a brightness-chromaticity loss that references the chromaticity of shadow-free regions, and an alignment loss to ensure smooth transitions between shadowed and shadow-free regions. Experimental results on the AISTD and SRD datasets demonstrate that our method achieves superior shadow removal performance.

Figures (5)

• Figure 1: Module heatmap comparison and PSNR analysis. The left (green) part shows the heatmap comparison of the basic generator framework after the Wavelet Attention Downsampling Module (WADM) is added ((b) and (c)). The right (red) section presents the wavelet transform of shadow and shadow-free images, highlighting low-frequency ((d) and (i)) and high-frequency components ((e-g) and (j-l)). PSNR shows that the D and V components have the highest similarity.
• Figure 2: Overall pipeline of our network. A network shadow and shadow-free domains use a shadow removal generator $G_s$ to transform $I_s$ to $O_{\text{sf}}$ and reconstruct $I_{\text{sf}}$. It features a WADM in the downsampling stage and proposes new losses like $\mathcal{L}_{\text{brightness-ch}}$, $\mathcal{L}_{\text{frequency}}$, and $\mathcal{L}_{\text{Align}}$ for enhanced shadow removal.
• Figure 3: Average brightness and illumination compensation. The lower left section is the average brightness pipeline where we process the L/B channels of the image in LAB space. After applying PCA and minimizing the entropy, we obtain $\sigma_{sf}^{em}$. Besides, illumination compensation is performed on the image to obtain $\sigma_{sf}^{IC}$ that is closer to the color brightness of the input image.
• Figure 4: (a) Input shadow image, (b) based on shadow-free chromaticity loss $\sigma_{s}^{\text{phy}}$, (c) Shadow brightness-chromaticity loss (ours) $\sigma_{sf}^{\text{IC}}$, (d) output shadow-free image, and (e) chromaticity map of the output image $\sigma_{sf}^{O}$. Compared with shadow-free chromaticity loss, our shadow brightness-chromaticity loss can make the shadow map closer to the shadow-free map, thus helping to remove shadows better.
• Figure 5: Shadow removal results. Comparison results on the soft shadow SRD dataset. (a) Input image, (b) Groundtruth, (c) Our result, (d) DC-shadowNet, (e) ST-CGAN, (f) DSC, (g) G2R-ShadowNet, and (h) Mask-ShadowGAN. Our unsupervised learning approach produces superior shadow-free results.