Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Data Poisoning against Differentially-Private Learners: Attacks and Defenses

Yuzhe Ma, Xiaojin Zhu, Justin Hsu

TL;DR

This work studies data poisoning against differentially private learners, revealing provable resistance when an attacker can modify only $k$ items but increasing vulnerability as $k$ grows. It introduces two attack paradigms—DPV (SGD-based on DP victims) and SV (surrogate-victim)—and instantiates them for objective- and output-perturbed logistic and ridge learners, deriving explicit gradient expressions via KKT conditions. Theoretical results provide lower bounds on attack effectiveness under $\epsilon$-DP (and $(\epsilon,\delta)$-DP) and are complemented by extensive experiments showing attacks become stronger with larger $k$ and weaker privacy, with deep-DPV often the most effective. The findings emphasize that while DP affords provable resistance, practical poisoning remains feasible under sufficient data modification, motivating tighter bounds and stronger defense strategies.

Abstract

Data poisoning attacks aim to manipulate the model produced by a learning algorithm by adversarially modifying the training set. We consider differential privacy as a defensive measure against this type of attack. We show that such learners are resistant to data poisoning attacks when the adversary is only able to poison a small number of items. However, this protection degrades as the adversary poisons more data. To illustrate, we design attack algorithms targeting objective and output perturbation learners, two standard approaches to differentially-private machine learning. Experiments show that our methods are effective when the attacker is allowed to poison sufficiently many training items.

Data Poisoning against Differentially-Private Learners: Attacks and Defenses

TL;DR

This work studies data poisoning against differentially private learners, revealing provable resistance when an attacker can modify only items but increasing vulnerability as grows. It introduces two attack paradigms—DPV (SGD-based on DP victims) and SV (surrogate-victim)—and instantiates them for objective- and output-perturbed logistic and ridge learners, deriving explicit gradient expressions via KKT conditions. Theoretical results provide lower bounds on attack effectiveness under -DP (and -DP) and are complemented by extensive experiments showing attacks become stronger with larger and weaker privacy, with deep-DPV often the most effective. The findings emphasize that while DP affords provable resistance, practical poisoning remains feasible under sufficient data modification, motivating tighter bounds and stronger defense strategies.

Abstract

Data poisoning attacks aim to manipulate the model produced by a learning algorithm by adversarially modifying the training set. We consider differential privacy as a defensive measure against this type of attack. We show that such learners are resistant to data poisoning attacks when the adversary is only able to poison a small number of items. However, this protection degrades as the adversary poisons more data. To illustrate, we design attack algorithms targeting objective and output perturbation learners, two standard approaches to differentially-private machine learning. Experiments show that our methods are effective when the attacker is allowed to poison sufficiently many training items.

Paper Structure

This paper contains 23 sections, 8 theorems, 54 equations, 6 figures.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Differential Privacy Resists Data Poisoning
  4. Data Poisoning Attacks on Private Learners
  5. SGD on Differentially-Private Victims (DPV)
  6. Instantiating Attack on Objective Perturbation
  7. Attacking Objective-Perturbed Logistic Regression.
  8. Attacking Objective-Perturbed Ridge Regression.
  9. Instantiating Attack on Output Perturbation
  10. Attacking Output-Perturbed Logistic Regression.
  11. Attacking Output-Perturbed Ridge Regression.
  12. SGD on Surrogate Victims (SV)
  13. Selecting Items to Poison
  14. Experiments
  15. Attacks Intelligently Modify Data
  16. ...and 8 more sections

Key Result

Theorem 1

Let $\mathcal{M}$ be an $\epsilon$-differentially-private learner. Let $J( \tilde{D})$ be the attack cost, where $\tilde{D}\in \mathcal{B}(D,k)$, then

Figures (6)

  • Figure 1: (a) 1D example (b, c) 2D example
  • Figure 2: poisoning trajectories in 2D.
  • Figure 3: (a-c) the attack cost $J(\tilde{D})$ decreases as $k$ grows. (d-f) the sparse attack trajectories for $k=10$
  • Figure 4: attack on objective and output-perturbed logistic regression.
  • Figure 5: attack on objective and output-perturbed ridge regression.
  • ...and 1 more figures

Theorems & Definitions (19)

  • Definition 1
  • Example 1
  • Example 2
  • Example 3
  • Theorem 1
  • proof
  • Lemma 1
  • proof
  • Corollary 1
  • proof
  • ...and 9 more