Active Honeypot Guardrail System: Probing and Confirming Multi-Turn LLM Jailbreaks
ChenYu Wu, Yi Wang, Yang Liao
TL;DR
The paper tackles multi-turn jailbreaks that defeat single-turn safety filters by introducing a honeypot guardrail that converts risk avoidance into risk exploitation. A Bait LLM generates non-executable, yet contextually relevant decoys, while a Response Filter safely rewrites outputs to remain useful yet non-actionable, enabling progressive intent revelation across turns. It formalizes the Honeypot Utility Score (HUS) combining Attractiveness and Feasibility with the Defense Efficacy Rate (DER) to balance safety and usability, and validates the approach on the MHJ dataset with ActorAttack, achieving a DER of 98.05% and low A/F scores (A ≈ 0.0818, F ≈ 0.0750), outperforming the native GPT-4o defense. The results demonstrate that proactive honeypot dynamics can effectively disrupt jailbreak progression while preserving benign user experience, suggesting practical avenues for adaptive, multi-layered defense and future improvements in adaptive honeypot generation and evaluation.
Abstract
Large language models (LLMs) are increasingly vulnerable to multi-turn jailbreak attacks, where adversaries iteratively elicit harmful behaviors that bypass single-turn safety filters. Existing defenses predominantly rely on passive rejection, which either fails against adaptive attackers or overly restricts benign users. We propose a honeypot-based proactive guardrail system that transforms risk avoidance into risk utilization. Our framework fine-tunes a bait model to generate ambiguous, non-actionable but semantically relevant responses, which serve as lures to probe user intent. Combined with the protected LLM's safe reply, the system inserts proactive bait questions that gradually expose malicious intent through multi-turn interactions. We further introduce the Honeypot Utility Score (HUS), measuring both the attractiveness and feasibility of bait responses, and use a Defense Efficacy Rate (DER) for balancing safety and usability. Initial experiment on MHJ Datasets with recent attack method across GPT-4o show that our system significantly disrupts jailbreak success while preserving benign user experience.