Adversarial examples for generative models

TL;DR

We address the vulnerability of deep generative models to adversarial inputs by developing three attack classes against VAEs and VAE-GANs. The classifier-augmented latent attack, the L_VAE-based attack, and the latent-space attack explore different attack vectors on reconstruction quality and latent representations. Evaluations on MNIST, SVHN, and CelebA show that latent-space attacks achieve the strongest targeted reconstructions, while classifier-based attacks can fool discriminators but often degrade reconstruction quality. These results underscore a broader vulnerability of generative models and motivate future defenses and robustness research.

Abstract

We explore methods of producing adversarial examples on deep generative models such as the variational autoencoder (VAE) and the VAE-GAN. Deep learning architectures are known to be vulnerable to adversarial examples, but previous work has focused on the application of adversarial examples to classification tasks. Deep generative models have recently become popular due to their ability to model input data distributions and generate realistic examples from those distributions. We present three classes of attacks on the VAE and VAE-GAN architectures and demonstrate them against networks trained on MNIST, SVHN and CelebA. Our first attack leverages classification-based adversaries by attaching a classifier to the trained encoder of the target generative model, which can then be used to indirectly manipulate the latent representation. Our second attack directly uses the VAE loss function to generate a target reconstruction image from the adversarial example. Our third attack moves beyond relying on classification or the standard loss for the gradient and directly optimizes against differences in source and target latent representations. We also motivate why an attacker might be interested in deploying such techniques against a target generative network.

Figures (22)

• Figure 1: Depiction of the attack scenario. The VAE is used as a compression scheme to transmit a latent representation of the image from the sender (left) to the receiver (right). The attacker convinces the sender to compress a particular image into its latent vector, which is sent to the receiver, where the decoder reconstructs the latent vector into some other image chosen by the attacker.
• Figure 2: Results for the $L_2$ optimization latent attack (see Section \ref{['sec:attack-direct']}) on the VAE-GAN, targeting a specific image from the class $0$. Shown are the first 12 non-zero images from the test SVHN data set. The columns are, in order: the original image, the reconstruction of the original image, the adversarial example, the predicted class of the adversarial example, the reconstruction of the adversarial example, the predicted class of the reconstructed adversarial example, the reconstruction of the reconstructed adversarial example (see Section \ref{['sec:measuring-effectiveness']}), and the predicted class of that reconstruction.
• Figure 3: The VAE-GAN classifier architecture used to generate classifier-based adversarial examples on the VAE-GAN. The VAE-GAN in the dashed box is the target network and is frozen while training the classifier. The path $\mathbf{x} \rightarrow f_{\mathrm{enc}} \rightarrow \mathbf{z} \rightarrow f_{\mathrm{class}} \rightarrow \hat{y}$ is used to generate adversarial examples in $\mathbf{z}$, which can then be reconstructed by $f_{\mathrm{dec}}$.
• Figure 4: Results for the $L_2$ optimization latent attack on the VAE-GAN, targeting the mean latent vector for $0$. Shown are the first 12 non-zero images from the test MNIST data set. The columns are, in order: the original image, the reconstruction of the original image, the adversarial example, the predicted class of the adversarial example, the reconstruction of the adversarial example, the predicted class of the reconstructed adversarial example, the reconstruction of the reconstructed adversarial example (see Section \ref{['sec:measuring-effectiveness']}), and the predicted class of that reconstruction.
• Figure 5: Left: representative adversarial examples with a target class of $0$ on the first $100$ non-zero images from the MNIST validation set. These were produced using the $L_2$ optimization latent attack (Section \ref{['sec:attack-direct']}). Middle: VAE-GAN reconstructions from adversarial examples produced using the $L_2$ optimization classifier attack on the same set of $100$ validation images (those adversaries are not shown, but are qualitatively similiar, see Section \ref{['sec:attack-indirect']}). Right: VAE-GAN reconstructions from the adversarial examples in the left column. Many of the classifier adversarial examples fail to reconstruct as zeros, whereas almost every adversarial example from the latent attack reconstructs as zero.