Indiscriminate Data Poisoning Attacks on Neural Networks

TL;DR

The paper addresses indiscriminate data poisoning of neural networks by casting the problem as a Stackelberg game and introducing the Total Gradient Descent Ascent (TGDA) method, which leverages second-order information to compute a total derivative and update a separate attacker model. This enables batch generation of thousands of poisoned points in one pass, significantly improving both effectiveness and efficiency over prior sequential approaches. Empirical results on MNIST and CIFAR-10 demonstrate that TGDA outperforms baselines across multiple architectures, while producing poisoned samples visually similar to clean data (clean-label). The work also analyzes transferability and defenses, revealing strengths and weaknesses of TGDA and highlighting practical considerations for robust defense design and future poisoning research.

Abstract

Data poisoning attacks, in which a malicious adversary aims to influence a model by injecting "poisoned" data into the training process, have attracted significant recent attention. In this work, we take a closer look at existing poisoning attacks and connect them with old and new algorithms for solving sequential Stackelberg games. By choosing an appropriate loss function for the attacker and optimizing with algorithms that exploit second-order information, we design poisoning attacks that are effective on neural networks. We present efficient implementations that exploit modern auto-differentiation packages and allow simultaneous and coordinated generation of tens of thousands of poisoned points, in contrast to existing methods that generate poisoned points one by one. We further perform extensive experiments that empirically explore the effect of data poisoning attacks on deep neural networks.

Figures (5)

• Figure 1: Comparing the efficacy of poisoning MNIST-17 with the PoisonSVM and Back-gradient attacks. The training set size is varied, while the ratio of the number of poisoned points to the training set size is fixed at $3\%$. These attacks become less effective as training set sizes increase.
• Figure 2: Our experimental protocol benchmarks data poisoning attacks. (1) Pretrain: the attacker and the defender are first trained on $\mathcal{D}_{tr}$ to yield a good autoencoder/classifier respectively. (2) During the attack, the attacker generates the optimal $\theta^*$ (thus $\mathcal{D}_{p}$) w.r.t $\mathcal{D}_{v}$ and the the optimal $\mathbf{w}^*$; the defender generates optimal $\mathbf{w}^*$ w.r.t $\mathcal{D}_{tr}' = \mathcal{D}_{tr} \cup \mathcal{D}_{p}$ and the optimal $\theta^*$ (which mimics testing).
• Figure 3: We visualize the poisoned data generated by the TGDA attack with/without pretraining the leader $\mathsf{L}$ on the MNIST dataset.
• Figure 4: Accuracy drop induced by our TGDA poisoning attack and baseline methods versus $\varepsilon$ (left three figures: MNIST; right two figures: CIFAR-10). Attack efficacy increases modestly with $\varepsilon$. Note that when $\varepsilon = 1$, only 50% of the training set is filled with poisoned data.
• Figure 5: We visualize the poisoned data generated by the TGDA attack and report the magnitude of perturbation (left: CIFAR-10; right: MNIST).