LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasures

TL;DR

The paper surveys security threats to large language models across their lifecycle, detailing training-time (causative) and inference-time (exploratory) attacks. It advances a two-pronged defense framework, separating prevention-based from detection-based approaches, and maps each defense to the attacks it mitigates, with nuanced assessments of effectiveness. Key contributions include a structured taxonomy of attacks (backdoors, data poisoning, gradient leakage, adversarial inputs, prompt hacking, and various inversion/memorization attacks) and a comprehensive catalog of defenses (paraphrasing, retokenization, delimiters, DP, anomaly detection, and more) along with their practical limits. The work offers a rigorous framework for securing LLMs and highlights gaps requiring further research to counter evolving threats in real-world deployments, including the balance between privacy, utility, and robustness. $D_{backdoor}$, $F$, and $x^*$ illustrate common attack models, while $(orall ext{adaptive adversaries})$ emphasizes the need for ongoing defense evolution.

Abstract

As large language models (LLMs) continue to evolve, it is critical to assess the security threats and vulnerabilities that may arise both during their training phase and after models have been deployed. This survey seeks to define and categorize the various attacks targeting LLMs, distinguishing between those that occur during the training phase and those that affect already trained models. A thorough analysis of these attacks is presented, alongside an exploration of defense mechanisms designed to mitigate such threats. Defenses are classified into two primary categories: prevention-based and detection-based defenses. Furthermore, our survey summarizes possible attacks and their corresponding defense strategies. It also provides an evaluation of the effectiveness of the known defense mechanisms for the different security threats. Our survey aims to offer a structured framework for securing LLMs, while also identifying areas that require further research to improve and strengthen defenses against emerging security challenges.