Deceptive Diffusion: Generating Synthetic Adversarial Examples

TL;DR

The paper addresses the vulnerability of diffusion-based generative systems to training-data poisoning by introducing deceptive diffusion, where a diffusion model trained on adversarial data can generate unlimited synthetic adversarial images. It combines adversarial attack algorithms with generative diffusion to produce misclassified images that do not correspond to any single original input, enabling scalable adversarial data for defense and exposing new security threats. Key findings include a deceptive diffusion model achieving a misclassification rate of $93.6\%$ on generated outputs and a linear degradation of classifier accuracy with increasing poisoning fraction $p$, as well as CAFD increasing with poisoning and mirroring PGDL2 attack patterns. The work highlights important implications for training-data integrity and proposes using adversarial data for robustness training while cautioning about novel attack vectors in diffusion-based generation, with broader relevance to healthcare and other domains where class balance is critical.

Abstract

We introduce the concept of deceptive diffusion -- training a generative AI model to produce adversarial images. Whereas a traditional adversarial attack algorithm aims to perturb an existing image to induce a misclassificaton, the deceptive diffusion model can create an arbitrary number of new, misclassified images that are not directly associated with training or test images. Deceptive diffusion offers the possibility of strengthening defence algorithms by providing adversarial training data at scale, including types of misclassification that are otherwise difficult to find. In our experiments, we also investigate the effect of training on a partially attacked data set. This highlights a new type of vulnerability for generative diffusion models: if an attacker is able to stealthily poison a portion of the training data, then the resulting diffusion model will generate a similar proportion of misleading outputs.

Figures (20)

• Figure 1: Confusion matrix for diffusion model trained on the 60,000 MNIST training images. With training images corresponding to each label (row) we show the frequency with which the classifier assigned each label (column). Entries on the diagonal therefore correspond to successfully created new images. Overall success rate is 99.5%.
• Figure 2: Building the deceptive diffusion model. Images that were successfully attacked by PGDL2 are used as training data, with the original labels retained. The trained diffusion model, $G_{\theta_{\text{final}}}$, produces adversarial images associated with a given a label. (For the images in this diagram, the image from PGDL2 is classified as an '8' and the image from the deceptive diffusion model is classified as a '5'.)
• Figure 3: Confusion matrix for the deceptive diffusion model. For a given label (row) we show the frequency with which the classifier assigned each label (column) to the output. Entries on the diagonal therefore correspond to unsuccessful attempts to create an adversarial image. Overall misclassification rate is 93.6%.
• Figure 4: Confusion matrix for PGDL2 attacks on the 60,000 MNIST training images. With training images corresponding to each label (row) we show the frequency with which the classifier assigned each label (column) after the attack. Entries on the diagonal therefore correspond to unsuccessful attacks. Overall success rate is 86.5%.
• Figure 5: Upper: example of 100 images arising when the deceptive diffusion model was given the label '9'. Lower: example of 100 images arising from successful PGDL2 attacks on images that had label '9'.