Logic Jailbreak: Efficiently Unlocking LLM Safety Restrictions Through Formal Logical Expression

TL;DR

LogiBreak exposes a semantic vulnerability in LLM safety by translating harmful natural-language prompts into formal first-order logic expressions, thereby shifting prompts into a logic-based distribution while preserving meaning. The method employs NL-FOL translation augmented with a contextual grounding phrase and an instructional tail to produce $x'_{harmful}$ that can bypass safety filters in a black-box setting. A multilingual, logic-based jailbreak dataset (English, Chinese, Dutch, Japanese, Spanish) and semantic evaluation across multiple judges demonstrate strong, language-robust ASR gains compared with six baselines, across open and closed models. The work highlights the insufficiency of token-level safety defenses and argues for semantic-level post-training alignment, with significant practical implications for deploying robust, safety-aware LLMs. Future work should expand multilingual coverage and incorporate semantic-safety mechanisms to close the gap exposed by logic-based jailbreaks.

Abstract

Despite substantial advancements in aligning large language models (LLMs) with human values, current safety mechanisms remain susceptible to jailbreak attacks. We hypothesize that this vulnerability stems from distributional discrepancies between alignment-oriented prompts and malicious prompts. To investigate this, we introduce LogiBreak, a novel and universal black-box jailbreak method that leverages logical expression translation to circumvent LLM safety systems. By converting harmful natural language prompts into formal logical expressions, LogiBreak exploits the distributional gap between alignment data and logic-based inputs, preserving the underlying semantic intent and readability while evading safety constraints. We evaluate LogiBreak on a multilingual jailbreak dataset spanning three languages, demonstrating its effectiveness across various evaluation settings and linguistic contexts.

Figures (14)

• Figure 1: A Venn diagram illustrating the hypothesis that model vulnerabilities arise from distributional differences between alignment and malicious prompts, highlighting how special token prompts, low-resource languages (LRLs) and multilingual prompts are more prone to jailbreak.
• Figure 2: t-SNE visualization of raw requests and their corresponding reformed versions in English. Blue points represent raw requests, while red points denote their reformed counterparts.
• Figure 3: Overview of LogiBreak and demonstration of logical translation across multiple languages.
• Figure 4: The ASR@5 of Logibreak across different categories of jailbreak requests. Full definitions of the abbreviations used in the figure can be found in the Appendix \ref{['sec:cat']}
• Figure 5: t-SNE projection of LLM2Vec embeddings computed for three inputs: the original natural-language request, its reformulation as a logical expression, and the whole input prompt.