Protecting Your LLMs with Information Bottleneck

TL;DR

This work introduces IBProtector, an information-bottleneck–based defense that compresses adversarial prompts into a compact, informative sub-prompt $X_{ ext{sub}}$ via a trainable extractor to guard LLMs against jailbreaks without modifying the target model. By upper-bounding $I(X; X_{ ext{sub}})$ with a KL-based loss and enforcing contiguity, the approach yields a tractable optimization objective that preserves the required predictive information, while a prediction-quantifier term encourages faithful responses with minimal information loss. Across token- and prompt-level jailbreaks, IBProtector yields substantial reductions in attack success rates and maintains benign answering performance, with demonstrated transferability to unseen attacks and models and robustness against adaptive strategies. The method offers a practical, transferable defense for improving LLM safety with limited computational overhead and without requiring changes to underlying models, while acknowledging limitations related to tokenizer dependence and interpretability of the extracted sub-prompts.

Abstract

The advent of large language models (LLMs) has revolutionized the field of natural language processing, yet they might be attacked to produce harmful content. Despite efforts to ethically align LLMs, these are often fragile and can be circumvented by jailbreaking attacks through optimized or manual adversarial prompts. To address this, we introduce the Information Bottleneck Protector (IBProtector), a defense mechanism grounded in the information bottleneck principle, and we modify the objective to avoid trivial solutions. The IBProtector selectively compresses and perturbs prompts, facilitated by a lightweight and trainable extractor, preserving only essential information for the target LLMs to respond with the expected answer. Moreover, we further consider a situation where the gradient is not visible to be compatible with any LLM. Our empirical evaluations show that IBProtector outperforms current defense methods in mitigating jailbreak attempts, without overly affecting response quality or inference speed. Its effectiveness and adaptability across various attack methods and target LLMs underscore the potential of IBProtector as a novel, transferable defense that bolsters the security of LLMs without requiring modifications to the underlying models.

Figures (8)

• Figure 1: (left) Normal jailbreak process attacks aligned LLMs, where red parts represent an example of adversarial prefix and suffix. (right) Our IBProtector extracts compression information related to expected responses to mitigate jailbreaking attacks on LLMs.
• Figure 2: The model framework of IBProtector, where fire and snowflake denote frozen and trained parameters, respectively, and the small language model is optional. Given an input prompt, the extractor can extract the most informative parts for the predictor to respond to.
• Figure 3: An example of the IBProtector. We use '.' perturbation in the adversarial prompt thus all uninformative tokens are replaced with '.'.
• Figure 4: IBProtector's extractor and Smooth LLMs defense results from other target models, where a lower ASR is better. IBProtector is interpreted as masking the most useless information, whereas Smooth is interpreted as randomizing masks where the number of copies is a way of ensemble masks.
• Figure 5: Ablation study of the PAIR attacks on Vicuna-13B.