OTI: A Model-free and Visually Interpretable Measure of Image Attackability

TL;DR

The paper tackles measuring image attackability without relying on a model proxy by introducing Object Texture Intensity (OTI), a proxy-free, visually interpretable metric that links attackability to the semantic object's texture via the definition $OTI(x)=\frac{1}{C\times H\times W}\|\text{object}(x)\odot(f*x)\|_{1}$ with Sobel filtering. It decomposes attackability into Object Area Ratio (OAR) and Image Texture Intensity (ITI), and shows that their combination yields OTI, which correlates with a image's distance to the decision boundary and its susceptibility to mid-/high-frequency perturbations. The authors provide theoretical motivation and comprehensive experiments across classification and segmentation tasks, multiple attack types, and dataset scales, demonstrating that OTI discriminates image attackability while remaining model-free and visually interpretable. This work offers a practical tool for active learning, adversarial training, and attack optimization, with a visual linkage between image content and robustness that can aid human understanding and decision-making.

Abstract

Despite the tremendous success of neural networks, benign images can be corrupted by adversarial perturbations to deceive these models. Intriguingly, images differ in their attackability. Specifically, given an attack configuration, some images are easily corrupted, whereas others are more resistant. Evaluating image attackability has important applications in active learning, adversarial training, and attack enhancement. This prompts a growing interest in developing attackability measures. However, existing methods are scarce and suffer from two major limitations: (1) They rely on a model proxy to provide prior knowledge (e.g., gradients or minimal perturbation) to extract model-dependent image features. Unfortunately, in practice, many task-specific models are not readily accessible. (2) Extracted features characterizing image attackability lack visual interpretability, obscuring their direct relationship with the images. To address these, we propose a novel Object Texture Intensity (OTI), a model-free and visually interpretable measure of image attackability, which measures image attackability as the texture intensity of the image's semantic object. Theoretically, we describe the principles of OTI from the perspectives of decision boundaries as well as the mid- and high-frequency characteristics of adversarial perturbations. Comprehensive experiments demonstrate that OTI is effective and computationally efficient. In addition, our OTI provides the adversarial machine learning community with a visual understanding of attackability.

Figures (12)

• Figure 1: Measuring image attackability by object texture intensity (values in the second row). The third and fourth rows are Grad-CAM visualizations of benign images and adversarial examples under BSR attack (surrogate: R50, target: D161). The last two rows show the predicted labels of adversarial examples and the predicted probabilities of the true class for both benign images and adversarial examples.
• Figure 2: Illustrations of how semantic object area ratio and texture intensity affect the robustness of benign samples. DeCoWA is used as the attack with R50 as the surrogate model. The target model is D161 in (a) and is SwinT in (b).
• Figure 3: Comparison of overall pipelines between conventional methods and our proposed method.
• Figure 4: Illustration explaining the effectiveness of OTI based on model decision boundaries.
• Figure 5: ASRs under varying sampling rates $\alpha$ in cross-model untargeted attacks. The surrogate models are ResNet-50 (top row) and ViT-B/16 (bottom row). The x-axis represents the target models, with the last one indicating the average.