StructuralSleight: Automated Jailbreak Attacks on Large Language Models Utilizing Uncommon Text-Organization Structures

TL;DR

Addressing the vulnerability of LLM safety alignment to manipulation via atypical prompt structures, the paper introduces Uncommon Text-Organization Structures (UTOS) and StructuralSleight, an automated, three-stage jailbreak framework. It demonstrates that structure-level prompts can bypass defenses that work on natural-language prompts, and shows synergistic effects when combining Structure Attack with obfuscation at the character and context levels, achieving up to 94.62% ASR on GPT-4o. The authors provide a taxonomy of 12 UTOS templates across four classes, a greedy stage-wise optimization strategy, and extensive experiments on six LLMs with AdvBench. They discuss defense directions, ethical considerations, and limitations, highlighting the need for structure-aware safety and future improvements in parsing, perplexity-based detection, and post-training defense. This work has practical implications for evaluating and hardening LLM safety against novel structure-based jailbreaks.

Abstract

Large Language Models (LLMs) are widely used in natural language processing but face the risk of jailbreak attacks that maliciously induce them to generate harmful content. Existing jailbreak attacks, including character-level and context-level attacks, mainly focus on the prompt of plain text without specifically exploring the significant influence of its structure. In this paper, we focus on studying how the prompt structure contributes to the jailbreak attack. We introduce a novel structure-level attack method based on long-tailed structures, which we refer to as Uncommon Text-Organization Structures (UTOS). We extensively study 12 UTOS templates and 6 obfuscation methods to build an effective automated jailbreak tool named StructuralSleight that contains three escalating attack strategies: Structural Attack, Structural and Character/Context Obfuscation Attack, and Fully Obfuscated Structural Attack. Extensive experiments on existing LLMs show that StructuralSleight significantly outperforms the baseline methods. In particular, the attack success rate reaches 94.62\% on GPT-4o, which has not been addressed by state-of-the-art techniques.

Figures (7)

• Figure 1: Motivating example: Jailbreak attempts on LLMs using different query methods. (a) Direct query; (b) GB18030 encoding; (c) Multi-task embedding; (d) Structured graph template. The structured approach (d) successfully bypasses safety measures, motivating our exploration of structure-level attack.
• Figure 2: An example for structural attack (simplified)
• Figure 3: Examples of Structural and Character/Context Obfuscation Attacks (simplified)
• Figure 4: Overview of StructuralSleight. StructuralSleight consists of 3 parts: SA, SCA and FSA. It employs a greedy strategy to select the optimal attack method for the subsequent stage.
• Figure 5: The ASR comparison of four UTOS templates in SA and SCA(GPT-4o)