Playing Language Game with LLMs Leads to Jailbreaking

TL;DR

This paper introduces two novel jailbreak methods based on mismatched generalization: natural language games and custom language games, both of which effectively bypass the safety mechanisms of LLMs, with various kinds and different variants, making them hard to defend and leading to high attack rates.

Abstract

The advent of large language models (LLMs) has spurred the development of numerous jailbreak techniques aimed at circumventing their security defenses against malicious attacks. An effective jailbreak approach is to identify a domain where safety generalization fails, a phenomenon known as mismatched generalization. In this paper, we introduce two novel jailbreak methods based on mismatched generalization: natural language games and custom language games, both of which effectively bypass the safety mechanisms of LLMs, with various kinds and different variants, making them hard to defend and leading to high attack rates. Natural language games involve the use of synthetic linguistic constructs and the actions intertwined with these constructs, such as the Ubbi Dubbi language. Building on this phenomenon, we propose the custom language games method: by engaging with LLMs using a variety of custom rules, we successfully execute jailbreak attacks across multiple LLM platforms. Extensive experiments demonstrate the effectiveness of our methods, achieving success rates of 93% on GPT-4o, 89% on GPT-4o-mini and 83% on Claude-3.5-Sonnet. Furthermore, to investigate the generalizability of safety alignments, we fine-tuned Llama-3.1-70B with the custom language games to achieve safety alignment within our datasets and found that when interacting through other language games, the fine-tuned models still failed to identify harmful content. This finding indicates that the safety alignment knowledge embedded in LLMs fails to generalize across different linguistic formats, thus opening new avenues for future research in this area.

Figures (16)

• Figure 1: An example of a jailbreak attack utilizing the language game Ubbi Dubbi (accessed in August 2024), in which the safety alignments fail to recognize the harmful intent of the question.
• Figure 2: The prompt templates, including both natural language games and custom language games.
• Figure 3: The jailbreak counts of six categories for each language game. Each set of columns in the histogram represents the counts of GPT-4o, GPT-4o-mini and Claude-3.5-Sonnet, respectively.
• Figure 4: The prompt for LLM evalution.
• Figure 5: Case for ubbi dubbi.