Game-Theoretic Unlearnable Example Generator

TL;DR

This work reframes unlearnable example attacks as a Stackelberg game between a data-poisoning attacker and a learning classifier, proving equilibria exist under broad conditions. It then proposes Game Unlearnable Example (GUE), which computes the game equilibrium using a first-order method and an encoder–decoder poison generator, with a surrogate loss to stabilize optimization. The authors demonstrate strong poisoning performance on CIFAR-10/100, showing that GUE generalizes to unseen data and remains effective under adversarial training and various defenses. The approach offers a principled, scalable framework for robust unlearnable data protection and provides valuable theoretical and empirical insights into game-theoretic data poisoning.

Abstract

Unlearnable example attacks are data poisoning attacks aiming to degrade the clean test accuracy of deep learning by adding imperceptible perturbations to the training samples, which can be formulated as a bi-level optimization problem. However, directly solving this optimization problem is intractable for deep neural networks. In this paper, we investigate unlearnable example attacks from a game-theoretic perspective, by formulating the attack as a nonzero sum Stackelberg game. First, the existence of game equilibria is proved under the normal setting and the adversarial training setting. It is shown that the game equilibrium gives the most powerful poison attack in that the victim has the lowest test accuracy among all networks within the same hypothesis space, when certain loss functions are used. Second, we propose a novel attack method, called the Game Unlearnable Example (GUE), which has three main gradients. (1) The poisons are obtained by directly solving the equilibrium of the Stackelberg game with a first-order algorithm. (2) We employ an autoencoder-like generative network model as the poison attacker. (3) A novel payoff function is introduced to evaluate the performance of the poison. Comprehensive experiments demonstrate that GUE can effectively poison the model in various scenarios. Furthermore, the GUE still works by using a relatively small percentage of the training data to train the generator, and the poison generator can generalize to unseen data well. Our implementation code can be found at https://github.com/hong-xian/gue.

Figures (5)

• Figure 1: An illustration of GUE attack, where the trained generator can generalize to unseen data well.
• Figure 2: Test accuracy curves of ResNet-18 trained on poisoned data from different unlearnable example attacks on CIFAR-10.
• Figure 3: Test accuracy of ResNet-18 trained on poisoned CIFAR-10, where the poison generator is trained on different percentage of training data.
• Figure 4: Test accuracy of ResNet-18 trained on poisoned CIFAR-100, where the poison generator is trained on different percentage of train data.
• Figure 5: The training curves of $||\nabla_{\theta} \mathcal{J}_a||_2$ and $\mathcal{J}_c(w, \theta)$ when we use different loss function in $\mathcal{J}_{a}$ to compute the equilibrium: $\mathcal{L}_{ce}, \mathcal{L}_{sur}$ and $\mathcal{L}_{ce}$ with gradient clipping. We take $\log_{10}$ of all values for better visualization.

Theorems & Definitions (21)

• Theorem 3
• proof : Proof sketch.
• Corollary 4
• Proposition 5
• Corollary 6
• Lemma 7
• Corollary 8
• Remark 9
• Lemma 10
• Lemma 11