Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AttackLLM: LLM-based Attack Pattern Generation for an Industrial Control System

Chuadhry Mujeeb Ahmed

TL;DR

This paper tackles the challenge of scarce ICS attack data by proposing AttackLLM, a multi-agent LLM framework that combines data-driven invariant inference with design-informed reasoning to automatically generate attack patterns. The approach uses two LLMs to extract invariants from operational data and system documentation, a validation step to ensure consistency, and a third party to validate and extend attack patterns, benchmarking against expert-designed attacks on the SWaT testbed. It reports 159 total attacks across six SWaT stages with 120 validated, including novel and stealthy vectors, and shows AttackLLM can reproduce nine of ten expert patterns while revealing cross-stage vulnerabilities. The work demonstrates scalable, testbed-free generation of high-quality attack patterns that can stress-test ICS anomaly detectors and advance proactive defense strategies, with potential extensions to other ICS domains and adversarial training.

Abstract

Malicious examples are crucial for evaluating the robustness of machine learning algorithms under attack, particularly in Industrial Control Systems (ICS). However, collecting normal and attack data in ICS environments is challenging due to the scarcity of testbeds and the high cost of human expertise. Existing datasets are often limited by the domain expertise of practitioners, making the process costly and inefficient. The lack of comprehensive attack pattern data poses a significant problem for developing robust anomaly detection methods. In this paper, we propose a novel approach that combines data-centric and design-centric methodologies to generate attack patterns using large language models (LLMs). Our results demonstrate that the attack patterns generated by LLMs not only surpass the quality and quantity of those created by human experts but also offer a scalable solution that does not rely on expensive testbeds or pre-existing attack examples. This multi-agent based approach presents a promising avenue for enhancing the security and resilience of ICS environments.

AttackLLM: LLM-based Attack Pattern Generation for an Industrial Control System

TL;DR

This paper tackles the challenge of scarce ICS attack data by proposing AttackLLM, a multi-agent LLM framework that combines data-driven invariant inference with design-informed reasoning to automatically generate attack patterns. The approach uses two LLMs to extract invariants from operational data and system documentation, a validation step to ensure consistency, and a third party to validate and extend attack patterns, benchmarking against expert-designed attacks on the SWaT testbed. It reports 159 total attacks across six SWaT stages with 120 validated, including novel and stealthy vectors, and shows AttackLLM can reproduce nine of ten expert patterns while revealing cross-stage vulnerabilities. The work demonstrates scalable, testbed-free generation of high-quality attack patterns that can stress-test ICS anomaly detectors and advance proactive defense strategies, with potential extensions to other ICS domains and adversarial training.

Abstract

Malicious examples are crucial for evaluating the robustness of machine learning algorithms under attack, particularly in Industrial Control Systems (ICS). However, collecting normal and attack data in ICS environments is challenging due to the scarcity of testbeds and the high cost of human expertise. Existing datasets are often limited by the domain expertise of practitioners, making the process costly and inefficient. The lack of comprehensive attack pattern data poses a significant problem for developing robust anomaly detection methods. In this paper, we propose a novel approach that combines data-centric and design-centric methodologies to generate attack patterns using large language models (LLMs). Our results demonstrate that the attack patterns generated by LLMs not only surpass the quality and quantity of those created by human experts but also offer a scalable solution that does not rely on expensive testbeds or pre-existing attack examples. This multi-agent based approach presents a promising avenue for enhancing the security and resilience of ICS environments.

Paper Structure

This paper contains 11 sections, 2 equations, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Background and System Description
  3. Large Language Models
  4. System Overview
  5. SWaT Testbed and Dataset Description
  6. Invariants in Industrial Control Systems
  7. Physical Invariant Inference and Validation
  8. Results
  9. Related Work
  10. Conclusions
  11. Tables for Attack Patterns

Figures (2)

  • Figure 1: Overview of the workflows. Two LLM models are used to obtain the control invariants. An LLM based agent validates the control invariants. Workflow2 presented as dashed lines, take the validated control invariants and outputs new attack patterns, which in turn are analysed in comparison with an human expert designed attacks.
  • Figure 2: Comparison between human-generated attacks from sridhar_dataset_paper, and AttackLLM generated attacks. Besides new attacks, we see that there is a huge overlap between human and AttackLLM attacks.