A Survey on Offensive AI Within Cybersecurity

TL;DR

This survey paper on offensive AI will comprehensively cover various aspects related to attacks against and using AI systems, including adversarial machine learning, attacks against AI models, infrastructure, and interfaces, along with offensive techniques.

Abstract

Artificial Intelligence (AI) has witnessed major growth and integration across various domains. As AI systems become increasingly prevalent, they also become targets for threat actors to manipulate their functionality for malicious purposes. This survey paper on offensive AI will comprehensively cover various aspects related to attacks against and using AI systems. It will delve into the impact of offensive AI practices on different domains, including consumer, enterprise, and public digital infrastructure. The paper will explore adversarial machine learning, attacks against AI models, infrastructure, and interfaces, along with offensive techniques like information gathering, social engineering, and weaponized AI. Additionally, it will discuss the consequences and implications of offensive AI, presenting case studies, insights, and avenues for further research.

Figures (9)

• Figure 1: An overview of the offensive AI domain.
• Figure 2: A simple illustration of the MLOps Cycle.
• Figure 3: Illustration of adversarial threats possible. It is crucial to note how threat actors can execute poisoning, evasion, and extraction attacks on ML models during both the training and inference phases.
• Figure 4: A LCA attack as shown by Efficient_Label_Contamination_Attacks_Against_Black_Box_Learning_Models. Solid lines are the model's decision boundaries, and the dashed lines are the attacker’s objective boundary. The bigger red (blue) points are originally blue (red) and are flipped by the attacker to induce misclassifications. The attacked data forms two clusters. If one identifies an attacked point, one may look into its adjacent points to enhance defence. Besides, attacked points are usually extreme points, suggesting that these are more likely targets than points near the centroid.
• Figure 5: Creation of an adversarial example to target the Ostrich class as shown by Adversarial_Examples_and_their_implications___Deep_Learning_bits. The article mentions other instances such as (a) Stealing the identity of someone by wearing special glasses, (b) Misleading a self-driving car by altering traffic signs, (c) Disguise a weapon to avoid video detection, and (d) Bypass audio or fingerprint identification.