Breaking Free: How to Hack Safety Guardrails in Black-Box Diffusion Models!

TL;DR

EvoSeed addresses robustness gaps by enabling black-box creation of natural adversarial samples using a conditional diffusion model and a classifier, optimized via CMA-ES over an initial seed with an $L_\infty$ constraint. It demonstrates that high-quality, photorealistic adversarial images can mislead classifiers while staying perceptually close to the original, raising safety concerns about diffusion-model deployments. The work provides both qualitative and quantitative evidence of vulnerabilities, including transferability across classifiers and potential misuse for bypassing safety checks, and establishes a baseline for evolutionary-strategy based adversarial search in unrestricted diffusion-model settings. Overall, EvoSeed offers a framework to stress-test and improve robustness of vision systems while highlighting urgent considerations for safety and policy in diffusion-based AI systems.

Abstract

Deep neural networks can be exploited using natural adversarial samples, which do not impact human perception. Current approaches often rely on deep neural networks' white-box nature to generate these adversarial samples or synthetically alter the distribution of adversarial samples compared to the training distribution. In contrast, we propose EvoSeed, a novel evolutionary strategy-based algorithmic framework for generating photo-realistic natural adversarial samples. Our EvoSeed framework uses auxiliary Conditional Diffusion and Classifier models to operate in a black-box setting. We employ CMA-ES to optimize the search for an initial seed vector, which, when processed by the Conditional Diffusion Model, results in the natural adversarial sample misclassified by the Classifier Model. Experiments show that generated adversarial images are of high image quality, raising concerns about generating harmful content bypassing safety classifiers. Our research opens new avenues to understanding the limitations of current safety mechanisms and the risk of plausible attacks against classifier systems using image generation. Project Website can be accessed at: https://shashankkotyan.github.io/EvoSeed.

Figures (12)

• Figure 1: Adversarial images created with EvoSeed are prime examples of how to deceive a range of classifiers tailored for various tasks. Note that, the generated natural adversarial images differ from non-adversarial ones, suggesting the adversarial images' unrestricted nature.
• Figure 2: Illustration of the EvoSeed framework to optimize initial seed vector $z$ to generate a natural adversarial sample. The Covariance Matrix Adaptation Evolution Strategy (CMA-ES) iteratively refines the initial seed vector $z$ and finds an adversarial initial seed vector $z'$. This adversarial seed vector $z'$ can then be utilized by the Conditional Diffusion Model $G$ to generate a natural adversarial sample $x$ capable of deceiving the Classifier Model $F$.
• Figure 3: Exemplar adversarial images generated for the Object Classification Task. We show that images that are aligned with the conditioning can be misclassified.
• Figure 4: We demonstrate a malicious use of EvoSeed to generate harmful content bypassing safety mechanisms. These adversarial images are misclassified as appropriate, highlighting better post-image generation checking for such generated images.
• Figure 5: We demonstrate an application of EvoSeed to misclassify the individual's ethnicity in the generated image. This raises concerns about misrepresenting a demographic group's representation estimated by such classifiers.