Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Survey on Adversarial Machine Learning for Code Data: Realistic Threats, Countermeasures, and Interpretations

Yulong Yang, Haoran Fan, Chenhao Lin, Qian Li, Zhengyu Zhao, Chao Shen, Xiaohong Guan

TL;DR

This study categorizes existing attack techniques into three types based on the CIA triad, and covers each type of risk, examining threat model categorization, attack techniques, and countermeasures, while also introducing novel perspectives on eXplainable AI (XAI) and exploring the interconnections between different risks.

Abstract

Code Language Models (CLMs) have achieved tremendous progress in source code understanding and generation, leading to a significant increase in research interests focused on applying CLMs to real-world software engineering tasks in recent years. However, in realistic scenarios, CLMs are exposed to potential malicious adversaries, bringing risks to the confidentiality, integrity, and availability of CLM systems. Despite these risks, a comprehensive analysis of the security vulnerabilities of CLMs in the extremely adversarial environment has been lacking. To close this research gap, we categorize existing attack techniques into three types based on the CIA triad: poisoning attacks (integrity \& availability infringement), evasion attacks (integrity infringement), and privacy attacks (confidentiality infringement). We have collected so far the most comprehensive (79) papers related to adversarial machine learning for CLM from the research fields of artificial intelligence, computer security, and software engineering. Our analysis covers each type of risk, examining threat model categorization, attack techniques, and countermeasures, while also introducing novel perspectives on eXplainable AI (XAI) and exploring the interconnections between different risks. Finally, we identify current challenges and future research opportunities. This study aims to provide a comprehensive roadmap for both researchers and practitioners and pave the way towards more reliable CLMs for practical applications.

A Survey on Adversarial Machine Learning for Code Data: Realistic Threats, Countermeasures, and Interpretations

TL;DR

This study categorizes existing attack techniques into three types based on the CIA triad, and covers each type of risk, examining threat model categorization, attack techniques, and countermeasures, while also introducing novel perspectives on eXplainable AI (XAI) and exploring the interconnections between different risks.

Abstract

Code Language Models (CLMs) have achieved tremendous progress in source code understanding and generation, leading to a significant increase in research interests focused on applying CLMs to real-world software engineering tasks in recent years. However, in realistic scenarios, CLMs are exposed to potential malicious adversaries, bringing risks to the confidentiality, integrity, and availability of CLM systems. Despite these risks, a comprehensive analysis of the security vulnerabilities of CLMs in the extremely adversarial environment has been lacking. To close this research gap, we categorize existing attack techniques into three types based on the CIA triad: poisoning attacks (integrity \& availability infringement), evasion attacks (integrity infringement), and privacy attacks (confidentiality infringement). We have collected so far the most comprehensive (79) papers related to adversarial machine learning for CLM from the research fields of artificial intelligence, computer security, and software engineering. Our analysis covers each type of risk, examining threat model categorization, attack techniques, and countermeasures, while also introducing novel perspectives on eXplainable AI (XAI) and exploring the interconnections between different risks. Finally, we identify current challenges and future research opportunities. This study aims to provide a comprehensive roadmap for both researchers and practitioners and pave the way towards more reliable CLMs for practical applications.

Paper Structure

This paper contains 31 sections, 5 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Study Organization
  3. Motivation and Risk Categorization
  4. Paper Collection and Selection
  5. Integrity & Availability Infringement: Poisoning Attacks
  6. Threat Models and Catogerizations
  7. Availability Poisoning Attacks
  8. Integrity Poisoning Attacks
  9. Poisoning for Good
  10. Countermeasures
  11. Integrity Infringement: Evasion Attacks
  12. Threat Models and Categorizations
  13. White-box Attacks
  14. Transfer-based Black-box Attacks
  15. Query-based Black-box Attacks
  16. ...and 16 more sections

Figures (5)

  • Figure 1: Organization of this survey.
  • Figure 2: Statistics of collected literature. "CS", "AI", "SE" are the short for "Computer Security", "Artificial Intelligence", and "Software Engineering".
  • Figure 3: Statistics of poisoning attacks literature of CLMs.
  • Figure 4: Statistics of evasion attacks literature of CLMs.
  • Figure 5: Statistics of privacy attacks literature of CLMs.