RefSR-Adv: Adversarial Attack on Reference-based Image Super-Resolution Models

TL;DR

RefSR-Adv reveals a security vulnerability in reference-based super-resolution by attacking the auxiliary input rather than the low-resolution image. The method perturbs only the reference path using a white-box PGD optimization, crafting $I_{adv}$ to maximally diverge from a pseudo-ground-truth baseline $I_{clean}$ under $||\delta||_\infty\le\epsilon$. Experiments across CNN, Transformer, and Mamba RefSR models on CUFED5, WR-SR, and DRefSR show widespread degradation, with attack success amplified when $I_{LR}$ and $I_{Ref}$ are similar. The work highlights the critical risk of relying on external references in RefSR and motivates defenses that robustify the reference pathway to ensure practical reliability.

Abstract

Single Image Super-Resolution (SISR) aims to recover high-resolution images from low-resolution inputs. Unlike SISR, Reference-based Super-Resolution (RefSR) leverages an additional high-resolution reference image to facilitate the recovery of high-frequency textures. However, existing research mainly focuses on backdoor attacks targeting RefSR, while the vulnerability of the adversarial attacks targeting RefSR has not been fully explored. To fill this research gap, we propose RefSR-Adv, an adversarial attack that degrades SR outputs by perturbing only the reference image. By maximizing the difference between adversarial and clean outputs, RefSR-Adv induces significant performance degradation and generates severe artifacts across CNN, Transformer, and Mamba architectures on the CUFED5, WR-SR, and DRefSR datasets. Importantly, experiments confirm a positive correlation between the similarity of the low-resolution input and the reference image and attack effectiveness, revealing that the model's over-reliance on reference features is a key security flaw. This study reveals a security vulnerability in RefSR systems, aiming to urge researchers to pay attention to the robustness of RefSR.

Figures (4)

• Figure 1: Overview of the RefSR-Adv attack framework. The procedure comprises two primary stages: (1) Baseline Generation, where the clean super-resolution output $I_{clean}$ is synthesized to serve as the pseudo ground-truth anchor; and (2) Adversarial Optimization, where a learnable perturbation $\delta$ is iteratively optimized within the auxiliary reference stream to maximize the output discrepancy, ultimately inducing severe textural artifacts in the final adversarial output $I_{adv}$.
• Figure 2: Visual results on CUFED5 dataset.
• Figure 3: Visual results on WR-SR dataset.
• Figure 4: Visual results on DRefSR dataset.