Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploring Vulnerabilities and Protections in Large Language Models: A Survey

Frank Weizhen Liu, Chenhui Hu

TL;DR

The paper surveys security vulnerabilities in large language models, focusing on Prompt Hacking (prompt injection and jailbreaking) and Adversarial Attacks (backdoors and data poisoning). It articulates the mechanisms of these attacks and reviews a range of defense strategies, including preprocessing, detection, white-box fine-tuning and embedding purification, clustering-based remedies, and black-box perturbation- and prompting-based defenses. Key contributions include cataloging attack vectors (e.g., Compositional Instruction Attacks, ProAttack, CUBE) and outlining defense frameworks (SmoothLLM, DiffPure, RAP, ONION) across open- and closed-source LLMs. The findings highlight the evolving threat landscape and emphasize the need for robust, adaptable security frameworks to ensure resilient real-world deployment of LLM systems.

Abstract

As Large Language Models (LLMs) increasingly become key components in various AI applications, understanding their security vulnerabilities and the effectiveness of defense mechanisms is crucial. This survey examines the security challenges of LLMs, focusing on two main areas: Prompt Hacking and Adversarial Attacks, each with specific types of threats. Under Prompt Hacking, we explore Prompt Injection and Jailbreaking Attacks, discussing how they work, their potential impacts, and ways to mitigate them. Similarly, we analyze Adversarial Attacks, breaking them down into Data Poisoning Attacks and Backdoor Attacks. This structured examination helps us understand the relationships between these vulnerabilities and the defense strategies that can be implemented. The survey highlights these security challenges and discusses robust defensive frameworks to protect LLMs against these threats. By detailing these security issues, the survey contributes to the broader discussion on creating resilient AI systems that can resist sophisticated attacks.

Exploring Vulnerabilities and Protections in Large Language Models: A Survey

TL;DR

The paper surveys security vulnerabilities in large language models, focusing on Prompt Hacking (prompt injection and jailbreaking) and Adversarial Attacks (backdoors and data poisoning). It articulates the mechanisms of these attacks and reviews a range of defense strategies, including preprocessing, detection, white-box fine-tuning and embedding purification, clustering-based remedies, and black-box perturbation- and prompting-based defenses. Key contributions include cataloging attack vectors (e.g., Compositional Instruction Attacks, ProAttack, CUBE) and outlining defense frameworks (SmoothLLM, DiffPure, RAP, ONION) across open- and closed-source LLMs. The findings highlight the evolving threat landscape and emphasize the need for robust, adaptable security frameworks to ensure resilient real-world deployment of LLM systems.

Abstract

As Large Language Models (LLMs) increasingly become key components in various AI applications, understanding their security vulnerabilities and the effectiveness of defense mechanisms is crucial. This survey examines the security challenges of LLMs, focusing on two main areas: Prompt Hacking and Adversarial Attacks, each with specific types of threats. Under Prompt Hacking, we explore Prompt Injection and Jailbreaking Attacks, discussing how they work, their potential impacts, and ways to mitigate them. Similarly, we analyze Adversarial Attacks, breaking them down into Data Poisoning Attacks and Backdoor Attacks. This structured examination helps us understand the relationships between these vulnerabilities and the defense strategies that can be implemented. The survey highlights these security challenges and discusses robust defensive frameworks to protect LLMs against these threats. By detailing these security issues, the survey contributes to the broader discussion on creating resilient AI systems that can resist sophisticated attacks.
Paper Structure (15 sections, 2 figures)

This paper contains 15 sections, 2 figures.

Table of Contents

  1. Introduction
  2. Prompt Hacking
  3. Prompt Injection
  4. Defense Against Prompt Injection
  5. Jailbreaking Attack
  6. Defense Against Jailbreaking Attack
  7. Adversarial Attack
  8. Backdoor Attack
  9. Defense against Backdoor Attack
  10. White-Box Defense Strategies
  11. Clustering-Based Approaches
  12. Black-Box Defense Strategies
  13. Data Poisoning Attack
  14. Defense Against Data Poisoning Attack
  15. Conclusion

Figures (2)

  • Figure 1: Attack and Defense Mechanisms in Large Language Models. This diagram outlines the various types of attacks on LLMs, including Prompt Hacking and Adversarial Attacks, as well as the corresponding defense mechanisms.
  • Figure 2: Jailbreak GPT-4o openai_gpt4o using role-play strategy, which is inspired by learnprompting_jailbreaking.