When LLM Meets DRL: Advancing Jailbreaking Efficiency via DRL-guided Search

TL;DR

This work reframes jailbreaking as a guided search problem and introduces RLbreaker, a deep reinforcement learning framework that learns to mutate jailbreak prompts via a set of mutators to reliably elicit harmful answers from black-box LLMs. Through a customized reward and PPO-based training, RLbreaker shows superior attack effectiveness and transferability across six models, and resilience against multiple defenses, with extensive ablation validating design choices. The results highlight the potential of DRL-driven prompt engineering to reveal alignment blind spots and inform defense strategies, while also addressing ethical safeguards and limitations. Overall, the paper demonstrates that guided, learning-based prompt synthesis can outperform stochastic genetic methods and in-context approaches in black-box jailbreaking scenarios.

Abstract

Recent studies developed jailbreaking attacks, which construct jailbreaking prompts to fool LLMs into responding to harmful questions. Early-stage jailbreaking attacks require access to model internals or significant human efforts. More advanced attacks utilize genetic algorithms for automatic and black-box attacks. However, the random nature of genetic algorithms significantly limits the effectiveness of these attacks. In this paper, we propose RLbreaker, a black-box jailbreaking attack driven by deep reinforcement learning (DRL). We model jailbreaking as a search problem and design an RL agent to guide the search, which is more effective and has less randomness than stochastic search, such as genetic algorithms. Specifically, we design a customized DRL system for the jailbreaking problem, including a novel reward function and a customized proximal policy optimization (PPO) algorithm. Through extensive experiments, we demonstrate that RLbreaker is much more effective than existing jailbreaking attacks against six state-of-the-art (SOTA) LLMs. We also show that RLbreaker is robust against three SOTA defenses and its trained agents can transfer across different LLMs. We further validate the key design choices of RLbreaker via a comprehensive ablation study.

Figures (9)

• Figure 1: Overview of RLbreaker.
• Figure 2: RLbreaker vs. baselines against defenses.
• Figure 2: Ablation study and sensitivity test results. The results of "token" are zeros.
• Figure 3: Guided vs. stochastic search in a grid search problem. Here we assume the initial point is the block in the bottom left corner and the goal is to reach the red block on the top right corner following a certain strategy. The guided search moves towards the target following a fixed direction (for example given by the gradient), while the stochastic search jumps across different sub-regions.
• Figure 4: Illustration of prompt structure & Toxicity score of testing questions.