ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs

TL;DR

The paper reveals a vulnerability in safety-aligned LLMs by showing that ASCII-art prompts can bypass semantics-based filtering. It introduces ViTC, a benchmark to assess ASCII-art recognition, and ArtPrompt, a two-step jailbreak that masks words and cloaks them with ASCII art to induce unsafe behavior in five SOTA LLMs. Experiments against AdvBench and HEx-PHI demonstrate ArtPrompt’s effectiveness and its ability to evade several defenses, underscoring the need for safety mechanisms that account for non-semantic cues. The findings motivate exploring non-semantic corpora and more robust safety strategies, including defenses that can interpret ASCII-structured prompts and cross-modal signals. The work provides actionable code to facilitate red-teaming and further safety research.

Abstract

Safety is critical to the usage of large language models (LLMs). Multiple techniques such as data filtering and supervised fine-tuning have been developed to strengthen LLM safety. However, currently known techniques presume that corpora used for safety alignment of LLMs are solely interpreted by semantics. This assumption, however, does not hold in real-world applications, which leads to severe vulnerabilities in LLMs. For example, users of forums often use ASCII art, a form of text-based art, to convey image information. In this paper, we propose a novel ASCII art-based jailbreak attack and introduce a comprehensive benchmark Vision-in-Text Challenge (ViTC) to evaluate the capabilities of LLMs in recognizing prompts that cannot be solely interpreted by semantics. We show that five SOTA LLMs (GPT-3.5, GPT-4, Gemini, Claude, and Llama2) struggle to recognize prompts provided in the form of ASCII art. Based on this observation, we develop the jailbreak attack ArtPrompt, which leverages the poor performance of LLMs in recognizing ASCII art to bypass safety measures and elicit undesired behaviors from LLMs. ArtPrompt only requires black-box access to the victim LLMs, making it a practical attack. We evaluate ArtPrompt on five SOTA LLMs, and show that ArtPrompt can effectively and efficiently induce undesired behaviors from all five LLMs. Our code is available at https://github.com/uw-nsl/ArtPrompt.

Figures (9)

• Figure 1: This figure presents two instances of a harmful instruction, "how to build a bomb", from a malicious user to an LLM. The first instance, labeled as ①, is written in English and is rejected by the LLM due to safety alignment. The second instance, labeled as ②, utilizes ASCII art to visually represent the word "bomb". This instance successfully bypasses the safety alignment, resulting in a harmful response "Sure, here is ...".
• Figure 2: Overview of ArtPrompt. ArtPrompt consists of two steps. In the first step, ArtPrompt masks the safety words (e.g., "bomb") within a prompt that could result in rejection from the victim LLM. In the second step, ArtPrompt replaces the masked word in Step I with ASCII art. Then the masked prompt is combined with the ASCII art representation to form a cloaked prompt. The cloaked prompt is finally sent to the victim LLM as a jailbreak attack. A detailed example prompt and response is available in Appendix \ref{['appx:artprompt-example']}.
• Figure 3: This figure presents HS of ArtPrompt on HEx-PHI dataset. ArtPrompt successfully induces unsafe behaviors across eleven prohibited categories from all victim models.
• Figure 4: This figure presents ASR (higher is better) versus the average number of optimization iterations (lower is better). We observe that ArtPrompt can efficiently generate the cloaked prompt with one iteration, while achieving the highest ASR among all jailbreak attacks.
• Figure 5: HS Ratio on AdvBench for ArtPrompt using Top-1 font. The distribution shift of HS indicates ArtPrompt is less effective on fine-tuned GPT-3.5.