Emoji-Based Jailbreaking of Large Language Models
M P V S Gopinadh, S Mahaboob Hussain
TL;DR
This work examines emoji-based jailbreaking as a prompt-level vulnerability in large language models, testing 50 emoji-augmented prompts across four open-source LLMs to assess jailbreak success, ethical compliance, and latency. Using a local Ollama deployment, the study reveals model-specific differences: two models show around 10% jailbreak success, one achieves 0% jailbreak with full ethical compliance, and overall inter-model differences are statistically significant ( $\chi^2=32.94$, $p<0.001$ ). The findings highlight gaps in current safety mechanisms when faced with emoji representations and underscore the need for emoji-aware defenses and robust, diverse adversarial evaluation pipelines. The work has practical implications for deploying LLMs in interactive systems, motivating pre-deployment emoji-augmented testing and the development of automated detection across multimodal prompt spaces.
Abstract
Large Language Models (LLMs) are integral to modern AI applications, but their safety alignment mechanisms can be bypassed through adversarial prompt engineering. This study investigates emoji-based jailbreaking, where emoji sequences are embedded in textual prompts to trigger harmful and unethical outputs from LLMs. We evaluated 50 emoji-based prompts on four open-source LLMs: Mistral 7B, Qwen 2 7B, Gemma 2 9B, and Llama 3 8B. Metrics included jailbreak success rate, safety alignment adherence, and latency, with responses categorized as successful, partial and failed. Results revealed model-specific vulnerabilities: Gemma 2 9B and Mistral 7B exhibited 10 % success rates, while Qwen 2 7B achieved full alignment (0% success). A chi-square test (chi^2 = 32.94, p < 0.001) confirmed significant inter-model differences. While prior works focused on emoji attacks targeting safety judges or classifiers, our empirical analysis examines direct prompt-level vulnerabilities in LLMs. The results reveal limitations in safety mechanisms and highlight the necessity for systematic handling of emoji-based representations in prompt-level safety and alignment pipelines.