Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

"Haet Bhasha aur Diskrimineshun": Phonetic Perturbations in Code-Mixed Hinglish to Red-Team LLMs

Darpan Aswal, Siddharth D Jaiswal

TL;DR

This work demonstrates that safety guardrails for multilingual multimodal LLMs can fail dramatically when faced with code-mixed prompts and phonetic perturbations. By combining code-mixing with phonetic misspellings (CMP) and extending jailbreak templates, the authors show high attack success rates for both text (~99%) and image (~78%) generation, with strong output relevance. They perform a comprehensive evaluation across multiple open- and closed-models, using diverse datasets and a robust interpretability analysis (Integrated Gradients) to reveal tokenization shifts as the root cause of bypassing filters. The study underscores the need for generalizable safety alignment that goes beyond template-based defenses, especially in real-world multilingual and multimodal contexts, and highlights directions for future work in language coverage, modalities, and tokenization-aware defenses.

Abstract

Recently released LLMs have strong multilingual \& multimodal capabilities. Model vulnerabilities are exposed using audits and red-teaming efforts. Existing efforts have focused primarily on the English language; thus, models continue to be susceptible to multilingual jailbreaking strategies, especially for multimodal contexts. In this study, we introduce a novel strategy that leverages code-mixing and phonetic perturbations to jailbreak LLMs for both text and image generation tasks. We also present an extension to a current jailbreak-template-based strategy and propose a novel template, showing higher effectiveness than baselines. Our work presents a method to effectively bypass safety filters in LLMs while maintaining interpretability by applying phonetic misspellings to sensitive words in code-mixed prompts. We achieve a 99\% Attack Success Rate for text generation and 78\% for image generation, with Attack Relevance Rate of 100\% for text generation and 96\% for image generation for the phonetically perturbed code-mixed prompts. Our interpretability experiments reveal that phonetic perturbations impact word tokenization, leading to jailbreak success. Our study motivates increasing the focus towards more generalizable safety alignment for multilingual multimodal models, especially in real-world settings wherein prompts can have misspelt words. \textit{\textbf{Warning: This paper contains examples of potentially harmful and offensive content.}}

"Haet Bhasha aur Diskrimineshun": Phonetic Perturbations in Code-Mixed Hinglish to Red-Team LLMs

TL;DR

This work demonstrates that safety guardrails for multilingual multimodal LLMs can fail dramatically when faced with code-mixed prompts and phonetic perturbations. By combining code-mixing with phonetic misspellings (CMP) and extending jailbreak templates, the authors show high attack success rates for both text (~99%) and image (~78%) generation, with strong output relevance. They perform a comprehensive evaluation across multiple open- and closed-models, using diverse datasets and a robust interpretability analysis (Integrated Gradients) to reveal tokenization shifts as the root cause of bypassing filters. The study underscores the need for generalizable safety alignment that goes beyond template-based defenses, especially in real-world multilingual and multimodal contexts, and highlights directions for future work in language coverage, modalities, and tokenization-aware defenses.

Abstract

Recently released LLMs have strong multilingual \& multimodal capabilities. Model vulnerabilities are exposed using audits and red-teaming efforts. Existing efforts have focused primarily on the English language; thus, models continue to be susceptible to multilingual jailbreaking strategies, especially for multimodal contexts. In this study, we introduce a novel strategy that leverages code-mixing and phonetic perturbations to jailbreak LLMs for both text and image generation tasks. We also present an extension to a current jailbreak-template-based strategy and propose a novel template, showing higher effectiveness than baselines. Our work presents a method to effectively bypass safety filters in LLMs while maintaining interpretability by applying phonetic misspellings to sensitive words in code-mixed prompts. We achieve a 99\% Attack Success Rate for text generation and 78\% for image generation, with Attack Relevance Rate of 100\% for text generation and 96\% for image generation for the phonetically perturbed code-mixed prompts. Our interpretability experiments reveal that phonetic perturbations impact word tokenization, leading to jailbreak success. Our study motivates increasing the focus towards more generalizable safety alignment for multilingual multimodal models, especially in real-world settings wherein prompts can have misspelt words. \textit{\textbf{Warning: This paper contains examples of potentially harmful and offensive content.}}

Paper Structure

This paper contains 31 sections, 1 equation, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Datasets, Models & Jailbreaks
  4. Datasets Benchmarked
  5. Models Evaluated
  6. Jailbreak Templates
  7. Experimental Methodology
  8. Generating Input Prompts
  9. Evaluation Metrics
  10. Interpreting Phonetic Perturbations
  11. Results & Observations
  12. Success of Red-teaming Approach (RQ.1)
  13. Relevance of Generated Outputs (RQ.2)
  14. Explaining Phonetic Perturbations (RQ.3)
  15. Red-teaming multimodal models (RQ.4)
  16. ...and 16 more sections

Figures (5)

  • Figure 1: An example red-teaming input using our code-mixed phonetic perturbation strategy.
  • Figure 2: Sequence Attribution Scores for inputs in English.
  • Figure 3: Sequence Attribution Scores for code-mixed inputs.
  • Figure 4: Sequence Attribution Scores for code-mixed inputs with phonetic perturbations.
  • Figure 5: Harmful image outputs generated by ChatGPT-4o-mini using our CMP prompts.