Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses

Micah Goldblum, Dimitris Tsipras, Chulin Xie, Xinyun Chen, Avi Schwarzschild, Dawn Song, Aleksander Madry, Bo Li, Tom Goldstein

TL;DR

This survey comprehensively catalogs dataset security threats in ML, focusing on training-only data poisoning and backdoor/trojan attacks, and surveys defenses that detect, repair, or prevent such attacks. It provides a unified taxonomy, contrasts threat models, and highlights practical defenses including robust aggregation, DP, and trigger-reconstruction techniques. The work also identifies open problems—from scalable, industrial-scale poisoning to persistent backdoors under transfer learning and federated learning—and emphasizes the need for benchmarks. Overall, the paper clarifies the security landscape of dataset curation and offers a roadmap for strengthening real-world ML systems against data-centric attacks.

Abstract

As machine learning systems grow in scale, so do their training data requirements, forcing practitioners to automate and outsource the curation of training data in order to achieve state-of-the-art performance. The absence of trustworthy human supervision over the data collection process exposes organizations to security vulnerabilities; training data can be manipulated to control and degrade the downstream behaviors of learned models. The goal of this work is to systematically categorize and discuss a wide range of dataset vulnerabilities and exploits, approaches for defending against these threats, and an array of open problems in this space. In addition to describing various poisoning and backdoor threat models and the relationships among them, we develop their unified taxonomy.

Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses

TL;DR

This survey comprehensively catalogs dataset security threats in ML, focusing on training-only data poisoning and backdoor/trojan attacks, and surveys defenses that detect, repair, or prevent such attacks. It provides a unified taxonomy, contrasts threat models, and highlights practical defenses including robust aggregation, DP, and trigger-reconstruction techniques. The work also identifies open problems—from scalable, industrial-scale poisoning to persistent backdoors under transfer learning and federated learning—and emphasizes the need for benchmarks. Overall, the paper clarifies the security landscape of dataset curation and offers a roadmap for strengthening real-world ML systems against data-centric attacks.

Abstract

As machine learning systems grow in scale, so do their training data requirements, forcing practitioners to automate and outsource the curation of training data in order to achieve state-of-the-art performance. The absence of trustworthy human supervision over the data collection process exposes organizations to security vulnerabilities; training data can be manipulated to control and degrade the downstream behaviors of learned models. The goal of this work is to systematically categorize and discuss a wide range of dataset vulnerabilities and exploits, approaches for defending against these threats, and an array of open problems in this space. In addition to describing various poisoning and backdoor threat models and the relationships among them, we develop their unified taxonomy.

Paper Structure

This paper contains 43 sections, 3 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Training-Only Attacks
  3. Applications of Data Poisoning
  4. Feature Collision Attacks
  5. Bilevel Optimization
  6. Label Flipping
  7. Influence Functions
  8. Online Poisoning
  9. Data Poisoning in Federated Learning
  10. Open Problems
  11. Backdoor Attacks
  12. Comparison to evasion attacks.
  13. Applications of Backdoor Attacks
  14. Object recognition and detection.
  15. Basic Backdoor Attacks
  16. ...and 28 more sections

Figures (4)

  • Figure 1: A taxonomy of training-only data poisoning attacks.
  • Figure 2: A taxonomy of backdoor attacks.
  • Figure 3: Backdoor attacks with different triggers: (a) a square pattern flips the identity of a stop-sign (gu2017badnetsgu2017badnets); (b) dead code as the trigger for source code modeling (ramakrishnan2020backdoorsramakrishnan2020backdoors).
  • Figure 4: A taxonomy of defenses against training-only and backdoor attacks.