Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Scalable Defense against In-the-wild Jailbreaking Attacks with Safety Context Retrieval

Taiye Chen, Zeming Wei, Ang Li, Yisen Wang

TL;DR

The paper addresses the vulnerability of large language models to jailbreaking attacks and the limited generalization of static defenses to in-the-wild threats. It proposes Safety Context Retrieval (SCR), a scalable defense that uses retrieval-augmented generation to dynamically inject safety demonstrations into prompts, enabling adaptation to new attacks with minimal data. Empirical results show SCR dramatically reduces attack success rates against both known and novel jailbreak techniques, while preserving natural task performance and enabling cross-model applicability. The findings suggest a practical, scalable paradigm for LLM safety in real-world deployments, with future work focusing on refining retrieval accuracy and reducing overhead.

Abstract

Large Language Models (LLMs) are known to be vulnerable to jailbreaking attacks, wherein adversaries exploit carefully engineered prompts to induce harmful or unethical responses. Such threats have raised critical concerns about the safety and reliability of LLMs in real-world deployment. While existing defense mechanisms partially mitigate such risks, subsequent advancements in adversarial techniques have enabled novel jailbreaking methods to circumvent these protections, exposing the limitations of static defense frameworks. In this work, we explore defending against evolving jailbreaking threats through the lens of context retrieval. First, we conduct a preliminary study demonstrating that even a minimal set of safety-aligned examples against a particular jailbreak can significantly enhance robustness against this attack pattern. Building on this insight, we further leverage the retrieval-augmented generation (RAG) techniques and propose Safety Context Retrieval (SCR), a scalable and robust safeguarding paradigm for LLMs against jailbreaking. Our comprehensive experiments demonstrate how SCR achieves superior defensive performance against both established and emerging jailbreaking tactics, contributing a new paradigm to LLM safety. Our code will be available upon publication.

Scalable Defense against In-the-wild Jailbreaking Attacks with Safety Context Retrieval

TL;DR

The paper addresses the vulnerability of large language models to jailbreaking attacks and the limited generalization of static defenses to in-the-wild threats. It proposes Safety Context Retrieval (SCR), a scalable defense that uses retrieval-augmented generation to dynamically inject safety demonstrations into prompts, enabling adaptation to new attacks with minimal data. Empirical results show SCR dramatically reduces attack success rates against both known and novel jailbreak techniques, while preserving natural task performance and enabling cross-model applicability. The findings suggest a practical, scalable paradigm for LLM safety in real-world deployments, with future work focusing on refining retrieval accuracy and reducing overhead.

Abstract

Large Language Models (LLMs) are known to be vulnerable to jailbreaking attacks, wherein adversaries exploit carefully engineered prompts to induce harmful or unethical responses. Such threats have raised critical concerns about the safety and reliability of LLMs in real-world deployment. While existing defense mechanisms partially mitigate such risks, subsequent advancements in adversarial techniques have enabled novel jailbreaking methods to circumvent these protections, exposing the limitations of static defense frameworks. In this work, we explore defending against evolving jailbreaking threats through the lens of context retrieval. First, we conduct a preliminary study demonstrating that even a minimal set of safety-aligned examples against a particular jailbreak can significantly enhance robustness against this attack pattern. Building on this insight, we further leverage the retrieval-augmented generation (RAG) techniques and propose Safety Context Retrieval (SCR), a scalable and robust safeguarding paradigm for LLMs against jailbreaking. Our comprehensive experiments demonstrate how SCR achieves superior defensive performance against both established and emerging jailbreaking tactics, contributing a new paradigm to LLM safety. Our code will be available upon publication.

Paper Structure

This paper contains 24 sections, 3 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Empirical Analysis on Retrieval-based Defenses
  4. Experiment design
  5. Overall results
  6. Brief Summary
  7. Safety Context Retrieval
  8. Algorithm formulation
  9. Discussion
  10. Experiment
  11. General Set-up
  12. Robustness to common attacks
  13. Defending against in-the-wild jailbreaking attacks
  14. Natural performance
  15. Empirical understandings
  16. ...and 9 more sections

Figures (3)

  • Figure 1: Overview of conventional static defense module and safety context retrieval.
  • Figure 2: Reduced attack success rate (ASR) with only a few safety contexts for different simulated in-the-wild jailbreaking attacks. Each line represents a LLM.
  • Figure 3: Retrieval Accuracy (RA) of the safety contexts designed for three novel attacks. Each line represents a fixed number of the safe context base, and the X-axis represents the number of safety contexts designed for the particular attack added into the base.