Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Can LLMs be Fooled? Investigating Vulnerabilities in LLMs

Sara Abdali, Jia He, CJ Barberan, Richard Anarfi

TL;DR

This paper surveys vulnerabilities in large language models (LLMs) across training and inference stages, identifying model-based, training-time, and inference-time threats. It systematically reviews attacks such as model extraction, leeching, imitation, data poisoning, backdoors, paraphrasing, prompt injection, and jailbreaking, and surveys mitigation strategies including SAME, watermarking, dataset curation, and defensive prompting. Central to the work are two cross-cutting approaches: Model Editing, to modify model behavior via gradient, weight, memory-based, and ensemble methods; and Chroma Teaming, a multi-team defense framework combining red, blue, green, and purple perspectives to improve resilience. The authors also discuss limitations of current defenses and propose concrete future directions to advance robust, secure LLM deployment in real-world settings.

Abstract

The advent of Large Language Models (LLMs) has garnered significant popularity and wielded immense power across various domains within Natural Language Processing (NLP). While their capabilities are undeniably impressive, it is crucial to identify and scrutinize their vulnerabilities especially when those vulnerabilities can have costly consequences. One such LLM, trained to provide a concise summarization from medical documents could unequivocally leak personal patient data when prompted surreptitiously. This is just one of many unfortunate examples that have been unveiled and further research is necessary to comprehend the underlying reasons behind such vulnerabilities. In this study, we delve into multiple sections of vulnerabilities which are model-based, training-time, inference-time vulnerabilities, and discuss mitigation strategies including "Model Editing" which aims at modifying LLMs behavior, and "Chroma Teaming" which incorporates synergy of multiple teaming strategies to enhance LLMs' resilience. This paper will synthesize the findings from each vulnerability section and propose new directions of research and development. By understanding the focal points of current vulnerabilities, we can better anticipate and mitigate future risks, paving the road for more robust and secure LLMs.

Can LLMs be Fooled? Investigating Vulnerabilities in LLMs

TL;DR

This paper surveys vulnerabilities in large language models (LLMs) across training and inference stages, identifying model-based, training-time, and inference-time threats. It systematically reviews attacks such as model extraction, leeching, imitation, data poisoning, backdoors, paraphrasing, prompt injection, and jailbreaking, and surveys mitigation strategies including SAME, watermarking, dataset curation, and defensive prompting. Central to the work are two cross-cutting approaches: Model Editing, to modify model behavior via gradient, weight, memory-based, and ensemble methods; and Chroma Teaming, a multi-team defense framework combining red, blue, green, and purple perspectives to improve resilience. The authors also discuss limitations of current defenses and propose concrete future directions to advance robust, secure LLM deployment in real-world settings.

Abstract

The advent of Large Language Models (LLMs) has garnered significant popularity and wielded immense power across various domains within Natural Language Processing (NLP). While their capabilities are undeniably impressive, it is crucial to identify and scrutinize their vulnerabilities especially when those vulnerabilities can have costly consequences. One such LLM, trained to provide a concise summarization from medical documents could unequivocally leak personal patient data when prompted surreptitiously. This is just one of many unfortunate examples that have been unveiled and further research is necessary to comprehend the underlying reasons behind such vulnerabilities. In this study, we delve into multiple sections of vulnerabilities which are model-based, training-time, inference-time vulnerabilities, and discuss mitigation strategies including "Model Editing" which aims at modifying LLMs behavior, and "Chroma Teaming" which incorporates synergy of multiple teaming strategies to enhance LLMs' resilience. This paper will synthesize the findings from each vulnerability section and propose new directions of research and development. By understanding the focal points of current vulnerabilities, we can better anticipate and mitigate future risks, paving the road for more robust and secure LLMs.
Paper Structure (34 sections, 1 figure)

This paper contains 34 sections, 1 figure.

Table of Contents

  1. Introduction
  2. Vulnerabilities of LLMs Against Adversarial Attacks
  3. Model-based Vulnerabilities
  4. Model Extraction Attacks
  5. Mitigation Strategies
  6. Model Leeching Attacks
  7. Mitigation Strategies
  8. Model Imitation
  9. Mitigation Strategies
  10. Training-Time Vulnerabilities
  11. Data Poisoning
  12. Mitigation Strategies
  13. Backdoor Attack
  14. Mitigation Strategies
  15. Inference-Time Vulnerabilities
  16. ...and 19 more sections

Figures (1)

  • Figure 1: A visual overview of the main vulnerabilities and mitigation strategies that we cover. We cover three vulnerability areas and two proposed mitigation strategies.