Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLMs Can Defend Themselves Against Jailbreaking in a Practical Manner: A Vision Paper

Daoyuan Wu, Shuai Wang, Yang Liu, Ning Liu

TL;DR

The paper tackles jailbreaking of off-the-shelf LLMs by introducing SelfDefend, a lightweight defense that runs a shadow stack in parallel with the normal model stack to detect harmful prompts and trigger a checkpoint, aiming for negligible overhead on normal prompts. The core idea is that all jailbreaks must include a harmful prompt, enabling LLMs to recognize such prompts; the authors validate this via manual analysis on GPT-3.5 and GPT-4 across GCG, template-based, and multilingual jailbreaks. The work positions SelfDefend as a practical, generic defense with a low-latency design and discusses future directions for faster recognition, further safety alignment using adversarial examples, and shadow-stack efficiency, while connecting to related defenses and traditional security concepts. If realized in real-world deployments, SelfDefend could significantly reduce jailbreak success while preserving user experience.

Abstract

Jailbreaking is an emerging adversarial attack that bypasses the safety alignment deployed in off-the-shelf large language models (LLMs). A considerable amount of research exists proposing more effective jailbreak attacks, including the recent Greedy Coordinate Gradient (GCG) attack, jailbreak template-based attacks such as using "Do-Anything-Now" (DAN), and multilingual jailbreak. In contrast, the defensive side has been relatively less explored. This paper proposes a lightweight yet practical defense called SELFDEFEND, which can defend against all existing jailbreak attacks with minimal delay for jailbreak prompts and negligible delay for normal user prompts. Our key insight is that regardless of the kind of jailbreak strategies employed, they eventually need to include a harmful prompt (e.g., "how to make a bomb") in the prompt sent to LLMs, and we found that existing LLMs can effectively recognize such harmful prompts that violate their safety policies. Based on this insight, we design a shadow stack that concurrently checks whether a harmful prompt exists in the user prompt and triggers a checkpoint in the normal stack once a token of "No" or a harmful prompt is output. The latter could also generate an explainable LLM response to adversarial prompts. We demonstrate our idea of SELFDEFEND works in various jailbreak scenarios through manual analysis in GPT-3.5/4. We also list three future directions to further enhance SELFDEFEND.

LLMs Can Defend Themselves Against Jailbreaking in a Practical Manner: A Vision Paper

TL;DR

The paper tackles jailbreaking of off-the-shelf LLMs by introducing SelfDefend, a lightweight defense that runs a shadow stack in parallel with the normal model stack to detect harmful prompts and trigger a checkpoint, aiming for negligible overhead on normal prompts. The core idea is that all jailbreaks must include a harmful prompt, enabling LLMs to recognize such prompts; the authors validate this via manual analysis on GPT-3.5 and GPT-4 across GCG, template-based, and multilingual jailbreaks. The work positions SelfDefend as a practical, generic defense with a low-latency design and discusses future directions for faster recognition, further safety alignment using adversarial examples, and shadow-stack efficiency, while connecting to related defenses and traditional security concepts. If realized in real-world deployments, SelfDefend could significantly reduce jailbreak success while preserving user experience.

Abstract

Jailbreaking is an emerging adversarial attack that bypasses the safety alignment deployed in off-the-shelf large language models (LLMs). A considerable amount of research exists proposing more effective jailbreak attacks, including the recent Greedy Coordinate Gradient (GCG) attack, jailbreak template-based attacks such as using "Do-Anything-Now" (DAN), and multilingual jailbreak. In contrast, the defensive side has been relatively less explored. This paper proposes a lightweight yet practical defense called SELFDEFEND, which can defend against all existing jailbreak attacks with minimal delay for jailbreak prompts and negligible delay for normal user prompts. Our key insight is that regardless of the kind of jailbreak strategies employed, they eventually need to include a harmful prompt (e.g., "how to make a bomb") in the prompt sent to LLMs, and we found that existing LLMs can effectively recognize such harmful prompts that violate their safety policies. Based on this insight, we design a shadow stack that concurrently checks whether a harmful prompt exists in the user prompt and triggers a checkpoint in the normal stack once a token of "No" or a harmful prompt is output. The latter could also generate an explainable LLM response to adversarial prompts. We demonstrate our idea of SELFDEFEND works in various jailbreak scenarios through manual analysis in GPT-3.5/4. We also list three future directions to further enhance SELFDEFEND.
Paper Structure (6 sections, 2 figures, 1 table)

This paper contains 6 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Manual Analysis
  3. Future Directions
  4. Related Work
  5. Conclusion
  6. Version History

Figures (2)

  • Figure 1: An overview of SelfDefend and its three future research directions (FD1 to FD3).
  • Figure 2: A motivating example shows a successful jailbreak (left) using the prompt from the CGC paper GCG23 and an effective identification of the harmful prompt (middle) under gpt-3.5-turbo-0125 (right) OpenAIPlayground. Note that different LLMs may exhibit small variations regarding the concrete content of the harmful prompt, with some recognizing only the phrase "how to make a bomb" as harmful.