Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly

Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Zhibo Sun, Yue Zhang

TL;DR

This survey addresses how large language models intersect with security and privacy by organizing a literature review around three pillars: beneficial (Good), potentially harmful (Bad), and intrinsic vulnerabilities and defenses (Ugly). It analyzes 281 papers to show that LLMs most often contribute positively to code and data security, while user-level attacks and model-inherent vulnerabilities remain prominent challenges. The authors synthesize defenses across model architecture, training, and inference, and identify gaps such as limited practical work on model/parameter extraction attacks and the need for safer instruction tuning. The study provides a roadmap for leveraging LLM strengths in security while advancing robust, privacy-preserving defenses for real-world deployments.

Abstract

Large Language Models (LLMs), such as ChatGPT and Bard, have revolutionized natural language understanding and generation. They possess deep language comprehension, human-like text generation capabilities, contextual awareness, and robust problem-solving skills, making them invaluable in various domains (e.g., search engines, customer support, translation). In the meantime, LLMs have also gained traction in the security community, revealing security vulnerabilities and showcasing their potential in security-related tasks. This paper explores the intersection of LLMs with security and privacy. Specifically, we investigate how LLMs positively impact security and privacy, potential risks and threats associated with their use, and inherent vulnerabilities within LLMs. Through a comprehensive literature review, the paper categorizes the papers into "The Good" (beneficial LLM applications), "The Bad" (offensive applications), and "The Ugly" (vulnerabilities of LLMs and their defenses). We have some interesting findings. For example, LLMs have proven to enhance code security (code vulnerability detection) and data privacy (data confidentiality protection), outperforming traditional methods. However, they can also be harnessed for various attacks (particularly user-level attacks) due to their human-like reasoning abilities. We have identified areas that require further research efforts. For example, Research on model and parameter extraction attacks is limited and often theoretical, hindered by LLM parameter scale and confidentiality. Safe instruction tuning, a recent development, requires more exploration. We hope that our work can shed light on the LLMs' potential to both bolster and jeopardize cybersecurity.

A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly

TL;DR

This survey addresses how large language models intersect with security and privacy by organizing a literature review around three pillars: beneficial (Good), potentially harmful (Bad), and intrinsic vulnerabilities and defenses (Ugly). It analyzes 281 papers to show that LLMs most often contribute positively to code and data security, while user-level attacks and model-inherent vulnerabilities remain prominent challenges. The authors synthesize defenses across model architecture, training, and inference, and identify gaps such as limited practical work on model/parameter extraction attacks and the need for safer instruction tuning. The study provides a roadmap for leveraging LLM strengths in security while advancing robust, privacy-preserving defenses for real-world deployments.

Abstract

Large Language Models (LLMs), such as ChatGPT and Bard, have revolutionized natural language understanding and generation. They possess deep language comprehension, human-like text generation capabilities, contextual awareness, and robust problem-solving skills, making them invaluable in various domains (e.g., search engines, customer support, translation). In the meantime, LLMs have also gained traction in the security community, revealing security vulnerabilities and showcasing their potential in security-related tasks. This paper explores the intersection of LLMs with security and privacy. Specifically, we investigate how LLMs positively impact security and privacy, potential risks and threats associated with their use, and inherent vulnerabilities within LLMs. Through a comprehensive literature review, the paper categorizes the papers into "The Good" (beneficial LLM applications), "The Bad" (offensive applications), and "The Ugly" (vulnerabilities of LLMs and their defenses). We have some interesting findings. For example, LLMs have proven to enhance code security (code vulnerability detection) and data privacy (data confidentiality protection), outperforming traditional methods. However, they can also be harnessed for various attacks (particularly user-level attacks) due to their human-like reasoning abilities. We have identified areas that require further research efforts. For example, Research on model and parameter extraction attacks is limited and often theoretical, hindered by LLM parameter scale and confidentiality. Safe instruction tuning, a recent development, requires more exploration. We hope that our work can shed light on the LLMs' potential to both bolster and jeopardize cybersecurity.
Paper Structure (52 sections, 4 figures, 3 tables)

This paper contains 52 sections, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Findings
  3. Contributions
  4. Roadmap
  5. Background
  6. Large Language Models (LLMs)
  7. Comparison of Popular LLMs
  8. Overview
  9. Scope
  10. The Research Questions
  11. Positive Impacts on Security and Privacy
  12. LLMs for Code Security
  13. Secure Coding (C)
  14. Test Case Generating (TCG)
  15. Vulnerable Code Detecting (RE)
  16. ...and 37 more sections

Figures (4)

  • Figure 1: An overview of our collected papers.
  • Figure 2: Taxonomy of Cyberattacks. The colored boxes represent attacks that have been demonstrated to be executable using LLMs, whereas the gray boxes indicate attacks that cannot be executed with LLMs.
  • Figure 3: Prevalence of the existing attacks
  • Figure 4: Taxonomy of Threats and the Defenses. The line represents a defense technique that can defend against either a specific attack or a group of attacks.