QueryAttack: Jailbreaking Aligned Large Language Models Using Structured Non-natural Query Language

TL;DR

Safety-aligned LLMs remain vulnerable to jailbreaks using structured non-natural query languages. QueryAttack converts malicious natural-language prompts into programming-language–style templates, bypassing defenses and achieving state-of-the-art attack success across popular models. The method comprises three steps: extracting query components, filling template-based non-natural languages, and in-context learning to map templates to natural-language responses. The work highlights the need for defenses that translate structured prompts into natural-language semantics before safety checks and offers a practical defense strategy.

Abstract

Recent advances in large language models (LLMs) have demonstrated remarkable potential in the field of natural language processing. Unfortunately, LLMs face significant security and ethical risks. Although techniques such as safety alignment are developed for defense, prior researches reveal the possibility of bypassing such defenses through well-designed jailbreak attacks. In this paper, we propose QueryAttack, a novel framework to examine the generalizability of safety alignment. By treating LLMs as knowledge databases, we translate malicious queries in natural language into structured non-natural query language to bypass the safety alignment mechanisms of LLMs. We conduct extensive experiments on mainstream LLMs, and the results show that QueryAttack not only can achieve high attack success rates (ASRs), but also can jailbreak various defense methods. Furthermore, we tailor a defense method against QueryAttack, which can reduce ASR by up to $64\%$ on GPT-4-1106. Our code is available at https://github.com/horizonsinzqs/QueryAttack.

Figures (12)

• Figure 1: Existing methods reveal a well-established phenomenon: malicious queries, when well encrypted, can bypass the security defenses of LLMs, leading them to generate encrypted harmful outputs. Unlike these methods, QueryAttack achieves jailbreak by translating malicious inputs into structured non-natural query languages and is capable of directly inducing LLMs to generate harmful content in natural language without the need for a de-translation step.
• Figure 2: The overview of QureyAttack. QueryAttack executes a three-step process to jailbreak the target LLM: 1). Extracting three key query components from the original query. 2). Filling the query template to get a query code. 3). Applying in-context learning to help the target LLM understand the natural semantics of the template and prompting the target LLM using the query code.
• Figure 3: Templates written in common programming language styles.
• Figure 4: Performance of different language styles.
• Figure 5: The attention score of natural language malicious query and QueryAttack.