SEGA: A Transferable Signed Ensemble Gaussian Black-Box Attack against No-Reference Image Quality Assessment Models

TL;DR

This work addresses the vulnerability of No-Reference Image Quality Assessment (NR-IQA) models to adversarial perturbations, focusing on the challenge of cross-model transferability in black-box settings. It introduces SEGA, a transferable black-box attack that generates adversarial perturbations by Gaussian-smoothing gradients from multiple source NR-IQA models and ensembling them to approximate the target gradient, followed by a perturbation-filtering step for imperceptibility. The authors prove an upper bound on the gradient-approximation error, $\|\hat{g}(x)-\nabla h(x)\| \le (L\sigma + C/\sigma)\sqrt{2}\frac{\Gamma((d+1)/2)}{\Gamma(d/2)}$, and empirically validate SEGA on CLIVE and KonIQ-10k, showing strong transferability across CNN- and transformer-based NR-IQA models. The results demonstrate SEGA's ability to degrade ranking and score-consistency while maintaining high perceptual similarity (SSIM around 0.85–0.9), highlighting its practical value for robustness evaluation and guiding defenses in NR-IQA systems.

Abstract

No-Reference Image Quality Assessment (NR-IQA) models play an important role in various real-world applications. Recently, adversarial attacks against NR-IQA models have attracted increasing attention, as they provide valuable insights for revealing model vulnerabilities and guiding robust system design. Some effective attacks have been proposed against NR-IQA models in white-box settings, where the attacker has full access to the target model. However, these attacks often suffer from poor transferability to unknown target models in more realistic black-box scenarios, where the target model is inaccessible. This work makes the first attempt to address the challenge of low transferability in attacking NR-IQA models by proposing a transferable Signed Ensemble Gaussian black-box Attack (SEGA). The main idea is to approximate the gradient of the target model by applying Gaussian smoothing to source models and ensembling their smoothed gradients. To ensure the imperceptibility of adversarial perturbations, SEGA further removes inappropriate perturbations using a specially designed perturbation filter mask. Experimental results on the CLIVE dataset demonstrate the superior transferability of SEGA, validating its effectiveness in enabling successful transfer-based black-box attacks against NR-IQA models.

Figures (6)

• Figure 1: Overview of the proposed SEGA method. SEGA leverages ensembled gradients from multiple source models with Gaussian smoothing to approximate the gradient of the target model. To enhance the imperceptibility of adversarial perturbations, SEGA designs a perturbation filter mask to remove inappropriate perturbations.
• Figure 2: An illustration of Gaussian smoothing is presented. The original function is defined as $f(x) = \sin(x) + 0.3 \cdot \sin(5x) + \eta$, where $\eta$ represents random noise drawn from a normal distribution $\mathcal{N}(0,0.1)$. Gaussian smoothing is applied with three different standard deviation parameters: $\sigma = {1, 10, 100}$. The results demonstrate that a small $\sigma$ preserves excessive detail from the original noise, while an excessively large $\sigma$ leads to oversmoothing and thereby loses important signal characteristics. Therefore, selecting an appropriate $\sigma$ value is crucial for effective Gaussian smoothing.
• Figure 3: The perturbation filtering pipeline employs two masks. The first mask, $M^{\mathcal{F}}$, removes unimportant components from the approximated gradient. The second mask, $M^{\text{JND}}$, is designed according to the JND map. It filters perturbations on pixels where the JND value is below a threshold $\epsilon$, ensuring perceptual imperceptibility.
• Figure 4: Visualization of adversarial examples generated by different attack methods against different target models on the CLIVE dataset, (a) HyperIQA, (b) DBCNN, (c) LinearityIQA, (d) LIQE. Please zoom in for a clearer view.
• Figure 5: A visualization result of adversarial examples generated with and without the use of perturbation filtering.

Theorems & Definitions (4)

• Theorem 1
• proof
• Theorem 2
• proof