HoneyTrap: Deceiving Large Language Model Attackers to Honeypot Traps with Resilient Multi-Agent Defense

TL;DR

HoneyTrap tackles the evolving threat of jailbreaking LLMs by deploying a coordinated, four-agent defense that deceives attackers, prolongs interactions, and drains adversarial resources. It introduces MTJ-Pro, a progressive multi-turn jailbreak benchmark with seven strategies and a benign counterpart, and two targeted metrics, Mislead Success Rate (MSR) and Attack Resource Consumption (ARC). Across four models, HoneyTrap reduces attack success rate (ASR) while elevating MSR and ARC, and it generalizes to adaptive single-turn attacks while providing explainable forensic analyses and minimal impact on benign dialogue quality. The work demonstrates that a dynamic, collaborative honeypot can robustly counter long-horizon adversaries without sacrificing normal user experience, offering practical defense guidance for real-world LLM deployments.

Abstract

Jailbreak attacks pose significant threats to large language models (LLMs), enabling attackers to bypass safeguards. However, existing reactive defense approaches struggle to keep up with the rapidly evolving multi-turn jailbreaks, where attackers continuously deepen their attacks to exploit vulnerabilities. To address this critical challenge, we propose HoneyTrap, a novel deceptive LLM defense framework leveraging collaborative defenders to counter jailbreak attacks. It integrates four defensive agents, Threat Interceptor, Misdirection Controller, Forensic Tracker, and System Harmonizer, each performing a specialized security role and collaborating to complete a deceptive defense. To ensure a comprehensive evaluation, we introduce MTJ-Pro, a challenging multi-turn progressive jailbreak dataset that combines seven advanced jailbreak strategies designed to gradually deepen attack strategies across multi-turn attacks. Besides, we present two novel metrics: Mislead Success Rate (MSR) and Attack Resource Consumption (ARC), which provide more nuanced assessments of deceptive defense beyond conventional measures. Experimental results on GPT-4, GPT-3.5-turbo, Gemini-1.5-pro, and LLaMa-3.1 demonstrate that HoneyTrap achieves an average reduction of 68.77% in attack success rates compared to state-of-the-art baselines. Notably, even in a dedicated adaptive attacker setting with intensified conditions, HoneyTrap remains resilient, leveraging deceptive engagement to prolong interactions, significantly increasing the time and computational costs required for successful exploitation. Unlike simple rejection, HoneyTrap strategically wastes attacker resources without impacting benign queries, improving MSR and ARC by 118.11% and 149.16%, respectively.

Figures (8)

• Figure 1: Illustration of the progressively intensifying multi-turn jailbreak attack and the multi-agent deceptive honeypot defense. The attacker issues multiple rounds of increasingly potent prompts to coerce the LLM into generating content that defames the President of the United States. In parallel, HoneyTrap progressively activates its deceptive defense mechanism, leveraging staged multi-agent intervention to detect, mislead, and contain the attack.
• Figure 2: Overview of HoneyTrap deceptive defense framework against multi-turn jailbreak attack. In this attack, the adversary starts with benign or obfuscated prompts, gradually revealing malicious intent across multiple turns, ultimately escalating to a full jailbreak. HoneyTrap utilizes four specialized agents to counteract this progression: Threat Interceptor, Misdirection Controller, System Harmonizer, and Forensic Tracker, each playing a key role in defending against this evolving threat.
• Figure 3: Type distribution of the multi-turn adversarial and benign dialogue corpora in MTJ-Pro dataset.
• Figure 4: Dialogue turn count distribution of the multi-turn adversarial and benign dialogue corpora in MTJ-Pro dataset.
• Figure 5: Comparison of misleading response evaluation methods: Dic-Judge vs. GPT-Judge.