Concept-based Adversarial Attack: a Probabilistic Perspective

TL;DR

This work proposes a concept-based adversarial attack framework that extends beyond single-image perturbations by adopting a probabilistic perspective and effectively preserve the underlying concept, while achieving higher attack efficiency.

Abstract

We propose a concept-based adversarial attack framework that extends beyond single-image perturbations by adopting a probabilistic perspective. Rather than modifying a single image, our method operates on an entire concept - represented by a distribution - to generate diverse adversarial examples. Preserving the concept is essential, as it ensures that the resulting adversarial images remain identifiable as instances of the original underlying category or identity. By sampling from this concept-based adversarial distribution, we generate images that maintain the original concept but vary in pose, viewpoint, or background, thereby misleading the classifier. Mathematically, this framework remains consistent with traditional adversarial attacks in a principled manner. Our theoretical and empirical results demonstrate that concept-based adversarial attacks yield more diverse adversarial examples and effectively preserve the underlying concept, while achieving higher attack efficiency.

Figures (6)

• Figure 1: Comparison of a single‐image adversarial attack (left) versus our proposed concept‐based adversarial attack (right). In both cases, adversarial examples $x_\text{adv}$ are drawn from the product of a distance distribution $p_\text{dis}$ and a victim distribution $p_\text{vic}$. On the left, $p_\text{dis}$ is centered on a single image $x_\text{ori}$, so its overlap with $p_\text{vic}$ is small. Consequently, adversarial examples that successfully fool the victim classifier typically lose the original image’s meaning, whereas those that preserve the original meaning fail to deceive the classifier. In contrast, on the right, $p_\text{dis}$ spans the original concept $\mathcal{C}_\text{ori}$, greatly increasing overlap with $p_\text{vic}$. As a result, the generated adversarial examples both maintain the concept’s meaning and easily deceive the classifier. (A green image border indicates an example that successfully fools the classifier; red indicates failure.)
• Figure 2: Illustration of how a single corgi concept ("[V] dog") is expanded into a diverse dataset. DreamBooth images (left) are finetuned with LoRA in Stable Diffusion XL, guided by GPT-4o prompts, to generate various poses, viewpoints, and environments (right).
• Figure 3: Qualitative comparison. (A green border indicates an example that successfully fools the classifier; red indicates failure.) See Appendix \ref{['app:morequal']} for a more detailed qualitative analysis.
• Figure 4: Screenshot of the user interface for user study.
• Figure 5: Qualitative comparison (zoomed in). (A green border indicates an example that successfully fools the classifier; red indicates failure.)

Theorems & Definitions (6)

• Theorem 1
• Theorem 2
• Theorem 1
• proof
• Theorem 2
• proof