Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Attack and defense techniques in large language models: A survey and new perspectives

Zhiyu Liao, Kang Chen, Yuanguo Lin, Kangkang Li, Yunxuan Liu, Hefeng Chen, Xingwang Huang, Yuanhui Yu

TL;DR

The paper addresses security challenges in large language models by presenting a taxonomy of attacks (adversarial prompt attacks, optimized attacks, application-level attacks, and model theft) and defenses (prevention- and detection-based). It surveys concrete techniques such as jailbreaking, prompt injection, prompt leakage, QROA, and model extraction, and discusses defense methods including SmoothLLM, System-Mode Self-Reminder, SPML, StruQ, Attention Tracker, and watermarking-based approaches. The study highlights open problems, such as adaptive scalable defenses, explainable security, and standardized evaluation frameworks, and stresses the need for interdisciplinary collaboration and ethical considerations in deploying secure LLMs. Overall, the work provides actionable insights and a comprehensive roadmap for developing robust and trustworthy LLM systems in real-world applications.

Abstract

Large Language Models (LLMs) have become central to numerous natural language processing tasks, but their vulnerabilities present significant security and ethical challenges. This systematic survey explores the evolving landscape of attack and defense techniques in LLMs. We classify attacks into adversarial prompt attack, optimized attacks, model theft, as well as attacks on application of LLMs, detailing their mechanisms and implications. Consequently, we analyze defense strategies, including prevention-based and detection-based defense methods. Although advances have been made, challenges remain to adapt to the dynamic threat landscape, balance usability with robustness, and address resource constraints in defense implementation. We highlight open problems, including the need for adaptive scalable defenses, explainable security techniques, and standardized evaluation frameworks. This survey provides actionable insights and directions for developing secure and resilient LLMs, emphasizing the importance of interdisciplinary collaboration and ethical considerations to mitigate risks in real-world applications.

Attack and defense techniques in large language models: A survey and new perspectives

TL;DR

The paper addresses security challenges in large language models by presenting a taxonomy of attacks (adversarial prompt attacks, optimized attacks, application-level attacks, and model theft) and defenses (prevention- and detection-based). It surveys concrete techniques such as jailbreaking, prompt injection, prompt leakage, QROA, and model extraction, and discusses defense methods including SmoothLLM, System-Mode Self-Reminder, SPML, StruQ, Attention Tracker, and watermarking-based approaches. The study highlights open problems, such as adaptive scalable defenses, explainable security, and standardized evaluation frameworks, and stresses the need for interdisciplinary collaboration and ethical considerations in deploying secure LLMs. Overall, the work provides actionable insights and a comprehensive roadmap for developing robust and trustworthy LLM systems in real-world applications.

Abstract

Large Language Models (LLMs) have become central to numerous natural language processing tasks, but their vulnerabilities present significant security and ethical challenges. This systematic survey explores the evolving landscape of attack and defense techniques in LLMs. We classify attacks into adversarial prompt attack, optimized attacks, model theft, as well as attacks on application of LLMs, detailing their mechanisms and implications. Consequently, we analyze defense strategies, including prevention-based and detection-based defense methods. Although advances have been made, challenges remain to adapt to the dynamic threat landscape, balance usability with robustness, and address resource constraints in defense implementation. We highlight open problems, including the need for adaptive scalable defenses, explainable security techniques, and standardized evaluation frameworks. This survey provides actionable insights and directions for developing secure and resilient LLMs, emphasizing the importance of interdisciplinary collaboration and ethical considerations to mitigate risks in real-world applications.
Paper Structure (30 sections, 5 equations, 5 figures, 2 tables)

This paper contains 30 sections, 5 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background on LLMs
  3. LLM Architecture and Workflow
  4. Current Status of LLM Security Research
  5. Attack Techniques
  6. Adversarial Prompt Attack
  7. Jailbreak
  8. Prompt Injection
  9. Prompt Leakage
  10. Optimized Attacks
  11. Attacks on Application of LLMs
  12. Model Theft
  13. Defense Techniques
  14. Defense against Jailbreak
  15. Prevention-Based Defense
  16. ...and 15 more sections

Figures (5)

  • Figure 1: Taxonomy of attack and defense techniques related to LLMs in our survey
  • Figure 2: The overall workflow of an LLM program
  • Figure 3: The objective of ABJ attack lin2024figure is achieved by generating customized data based on malicious inputs and then analyzing the generated data by instructing the target model. AutoDAN liu2023autodan utilizes internal evaluation and genetic algorithms to determine the optimal template for completing the entire attack process.
  • Figure 4: The general workflow of SPMLsharma2024spml involves compiling the input prompts using this method and then checking the compiled results based on specific methods to ensure the security of the input prompts.
  • Figure 5: Watermark Formation: The imitation model is embedded with a similar API to that of the victim by incorporating the victim's API into the output of the model. Watermark Detection: A specific algorithm is used to detect watermark signals from the imitation model.