Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Benchmarking Adversarial Robustness of Image Shadow Removal with Shadow-adaptive Attacks

Chong Wang, Yi Yu, Lanqing Guo, Bihan Wen

TL;DR

The paper tackles adversarial robustness in image shadow removal by introducing a shadow-adaptive adversarial attack that allocates perturbation budgets proportionally to pixel intensities to respect the illumination heterogeneity between shadow and non-shadow regions. It defines and analyzes the attack with per-pixel constraints, leveraging PGD and a normalization constraint, and proves an $ ext{L}_1$-norm equivalence between adaptive and uniform budgets via $\epsilon_u = \epsilon_a \bar{I}$. Through extensive experiments on ISTD and ISTD+ with five shadow-removal models, it demonstrates that the adaptive attack can degrade performance effectively while maintaining perturbation stealth, highlighting region-specific robustness differences and artifacts in some models. The work provides both theoretical and empirical benchmarks to guide the design of more robust shadow-removal methods in real-world scenarios where illumination varies spatially across images.

Abstract

Shadow removal is a task aimed at erasing regional shadows present in images and reinstating visually pleasing natural scenes with consistent illumination. While recent deep learning techniques have demonstrated impressive performance in image shadow removal, their robustness against adversarial attacks remains largely unexplored. Furthermore, many existing attack frameworks typically allocate a uniform budget for perturbations across the entire input image, which may not be suitable for attacking shadow images. This is primarily due to the unique characteristic of spatially varying illumination within shadow images. In this paper, we propose a novel approach, called shadow-adaptive adversarial attack. Different from standard adversarial attacks, our attack budget is adjusted based on the pixel intensity in different regions of shadow images. Consequently, the optimized adversarial noise in the shadowed regions becomes visually less perceptible while permitting a greater tolerance for perturbations in non-shadow regions. The proposed shadow-adaptive attacks naturally align with the varying illumination distribution in shadow images, resulting in perturbations that are less conspicuous. Building on this, we conduct a comprehensive empirical evaluation of existing shadow removal methods, subjecting them to various levels of attack on publicly available datasets.

Benchmarking Adversarial Robustness of Image Shadow Removal with Shadow-adaptive Attacks

TL;DR

The paper tackles adversarial robustness in image shadow removal by introducing a shadow-adaptive adversarial attack that allocates perturbation budgets proportionally to pixel intensities to respect the illumination heterogeneity between shadow and non-shadow regions. It defines and analyzes the attack with per-pixel constraints, leveraging PGD and a normalization constraint, and proves an -norm equivalence between adaptive and uniform budgets via . Through extensive experiments on ISTD and ISTD+ with five shadow-removal models, it demonstrates that the adaptive attack can degrade performance effectively while maintaining perturbation stealth, highlighting region-specific robustness differences and artifacts in some models. The work provides both theoretical and empirical benchmarks to guide the design of more robust shadow-removal methods in real-world scenarios where illumination varies spatially across images.

Abstract

Shadow removal is a task aimed at erasing regional shadows present in images and reinstating visually pleasing natural scenes with consistent illumination. While recent deep learning techniques have demonstrated impressive performance in image shadow removal, their robustness against adversarial attacks remains largely unexplored. Furthermore, many existing attack frameworks typically allocate a uniform budget for perturbations across the entire input image, which may not be suitable for attacking shadow images. This is primarily due to the unique characteristic of spatially varying illumination within shadow images. In this paper, we propose a novel approach, called shadow-adaptive adversarial attack. Different from standard adversarial attacks, our attack budget is adjusted based on the pixel intensity in different regions of shadow images. Consequently, the optimized adversarial noise in the shadowed regions becomes visually less perceptible while permitting a greater tolerance for perturbations in non-shadow regions. The proposed shadow-adaptive attacks naturally align with the varying illumination distribution in shadow images, resulting in perturbations that are less conspicuous. Building on this, we conduct a comprehensive empirical evaluation of existing shadow removal methods, subjecting them to various levels of attack on publicly available datasets.
Paper Structure (8 sections, 1 theorem, 4 equations, 3 figures, 1 table)

This paper contains 8 sections, 1 theorem, 4 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Adversarial Attacks for Shadow Removal Tasks
  3. Shadow-Adaptive Adversarial Attack
  4. Experiments
  5. Implementation Details
  6. Shadow-adaptive Attack vs. Uniform Attack
  7. Benchmarking the Adversarial Robustness
  8. Conclusion

Key Result

Theorem 1

Let $\epsilon_{a}$ denotes the attack budget in our proposed shadow-adaptive attacks in eq_adap_delta, we can set the budget $\epsilon_{u} = \epsilon_a \bar{I}$ for standard uniform attack in eq_ori_delta to achieve an equivalent attack strength, where $\bar{I}$ is the mean value of the input shado

Figures (3)

  • Figure 1: Comparison of the adversarial examples from (a) a shadow image from the ISTD+ dataset, using (b) the standard uniform attack, and (c) the proposed shadow-adaptive attack.(d) The shadow mask indicates that the average pixel intensity within the shadow and non-shadow regions, denoted as $\bar{I}_s$ and $\bar{I}_{ns}$, are significantly different. (e) and (f) show the normalized perturbation $\delta / I$ generated by the two attacks, which better reflect the visual sensitivity of the adversarial perturbations. The zoomed-in subfigures in (a)-(c) and the normalized perturbations in (e)$\&$(f) are multiplied by $3\times$ for better visibility.
  • Figure 2: Adversarial robustness of five existing shadow removal models against our proposed shadow-adaptive attack with various attack budget $\epsilon_a$ evaluated by PSNR and SSIM on ISTD wang2018stacked and ISTD+ le2019shadow datasets. Each subfigure in the dashed line indicates the results of the whole image, shadow region, and non-shadow region, respectively.
  • Figure 3: Visual comparison under our shadow-adaptive attack with budget $\epsilon_a$ = 16/255 on the ISTD dataset wang2018stacked (top row) and ISTD+ dataset le2019shadow (bottom row). Subfigures (i), (ii), and (iii) represent attacked shadow removal results, perturbed input shadow images, and original shadow removal results, respectively.

Theorems & Definitions (2)

  • Theorem 1
  • proof