Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Look Before You Leap: Enhancing Attention and Vigilance Regarding Harmful Content with GuidelineLLM

Shaoqing Zhang, Zhuosheng Zhang, Kehai Chen, Rongxiang Weng, Muyun Yang, Tiejun Zhao, Min Zhang

TL;DR

This work tackles the rising threat of jailbreak attacks on LLMs by introducing GuidelineLLM, a defense that identifies risky queries, summarises them into guidelines, and feeds these guidelines to the responder LLM before generating a reply. Unlike retraining the responder models, GuidelineLLM requires fine-tuning only on its own framework, enabling broad applicability across different LLMs. Empirical results show an average ASR reduction of 34.17% while maintaining helpfulness on benign queries, with strong performance against strong jailbreak methods and in-the-wild datasets. The approach combines a data-generation loop (T-Jailbreak) with a balanced guideline dataset and evaluates across multiple models, demonstrating significant safety gains with modest inference-time overhead and no safety fine-tuning of the responding LLMs.

Abstract

Despite being empowered with alignment mechanisms, large language models (LLMs) are increasingly vulnerable to emerging jailbreak attacks that can compromise their alignment mechanisms. This vulnerability poses significant risks to real-world applications. Existing work faces challenges in both training efficiency and generalization capabilities (i.e., Reinforcement Learning from Human Feedback and Red-Teaming). Developing effective strategies to enable LLMs to resist continuously evolving jailbreak attempts represents a significant challenge. To address this challenge, we propose a novel defensive paradigm called GuidelineLLM, which assists LLMs in recognizing queries that may have harmful content. Before LLMs respond to a query, GuidelineLLM first identifies potential risks associated with the query, summarizes these risks into guideline suggestions, and then feeds these guidelines to the responding LLMs. Importantly, our approach eliminates the necessity for additional safety fine-tuning of the LLMs themselves; only the GuidelineLLM requires fine-tuning. This characteristic enhances the general applicability of GuidelineLLM across various LLMs. Experimental results demonstrate that GuidelineLLM can significantly reduce the attack success rate (ASR) against LLM (an average reduction of 34.17\% ASR) while maintaining the usefulness of LLM in handling benign queries. The code is available at https://github.com/sqzhang-lazy/GuidelineLLM.

Look Before You Leap: Enhancing Attention and Vigilance Regarding Harmful Content with GuidelineLLM

TL;DR

This work tackles the rising threat of jailbreak attacks on LLMs by introducing GuidelineLLM, a defense that identifies risky queries, summarises them into guidelines, and feeds these guidelines to the responder LLM before generating a reply. Unlike retraining the responder models, GuidelineLLM requires fine-tuning only on its own framework, enabling broad applicability across different LLMs. Empirical results show an average ASR reduction of 34.17% while maintaining helpfulness on benign queries, with strong performance against strong jailbreak methods and in-the-wild datasets. The approach combines a data-generation loop (T-Jailbreak) with a balanced guideline dataset and evaluates across multiple models, demonstrating significant safety gains with modest inference-time overhead and no safety fine-tuning of the responding LLMs.

Abstract

Despite being empowered with alignment mechanisms, large language models (LLMs) are increasingly vulnerable to emerging jailbreak attacks that can compromise their alignment mechanisms. This vulnerability poses significant risks to real-world applications. Existing work faces challenges in both training efficiency and generalization capabilities (i.e., Reinforcement Learning from Human Feedback and Red-Teaming). Developing effective strategies to enable LLMs to resist continuously evolving jailbreak attempts represents a significant challenge. To address this challenge, we propose a novel defensive paradigm called GuidelineLLM, which assists LLMs in recognizing queries that may have harmful content. Before LLMs respond to a query, GuidelineLLM first identifies potential risks associated with the query, summarizes these risks into guideline suggestions, and then feeds these guidelines to the responding LLMs. Importantly, our approach eliminates the necessity for additional safety fine-tuning of the LLMs themselves; only the GuidelineLLM requires fine-tuning. This characteristic enhances the general applicability of GuidelineLLM across various LLMs. Experimental results demonstrate that GuidelineLLM can significantly reduce the attack success rate (ASR) against LLM (an average reduction of 34.17\% ASR) while maintaining the usefulness of LLM in handling benign queries. The code is available at https://github.com/sqzhang-lazy/GuidelineLLM.

Paper Structure

This paper contains 23 sections, 2 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Enhancing Safety Policies of LLMs
  4. Enhancing Inference Safety of LLMs
  5. The Proposed Method
  6. Initializing T-Jailbreak Data
  7. Initializing Guideline Data
  8. Fine-tuning GuidelineLLM Framework
  9. Fine-tuning and Inference
  10. Evaluating the Quality
  11. Experiments
  12. Dataset and Setup
  13. Evaluation Metrics
  14. Baseline
  15. Main Results
  16. ...and 8 more sections

Figures (2)

  • Figure 1: The comparison of the defensive policies between GuidelineLLM and typical alignment mechanisms. Alignment mechanisms require large-scale data and substantial computational resources to train responding LLMs. However, our GuidelineLLM does not require additional safety training for the responding LLMs.
  • Figure 2: The framework of fine-tuning GuidelineLLM. The framework is a semi-automatic approach. First, we use gpt-3.5-turbo-0125 to initialize our T-Jailbreak and Guideline Sets. Then, we use the collected T-Jailbreak and Guideline data to fine-tune the respective JailbreakLLM and GuidelineLLM. Next, we utilize JailbreakLLM to generate new jailbreak queries and filter out high-quality ones, using GuidelineLLM to generate the corresponding jailbreak guidelines. We also sample benign queries to generate guidelines and filter the successful, helpful examples. Finally, we use the updated T-Jailbreak and Guideline sets to iterate our JailbreakLLM and GuidelineLLM.