X-Guard: Multilingual Guard Agent for Content Moderation

TL;DR

The paper tackles multilingual safety gaps in LLM guardrails by introducing X-Guard, a transparent multilingual safety agent with a three-module pipeline: language detection, translation via a fine-tuned $132$-language translator, and a safety evaluator trained through supervised fine-tuning and GRPO. It addresses data scarcity and bias by curating open-source safety datasets with explicit reasoning, employing a jury of judges, and generating a $5{,}000{,}000$ translation data corpus across $132$ languages. Empirical evaluations show the agent achieves $70.38 ext{%}$ accuracy in safety labeling across languages (F1 $70.44 ext{%}$) and $52.37 ext{%}$ weighted F1 for category identification, with English-only performance near $97.20 ext{%}$ accuracy. The model also defends against code-switching attacks (Sandwich Attack) with $83 ext{%}$ accuracy, outperforming non-ensemble multilingual guards, and is complemented by a discussion of limitations and future directions for broader coverage and robustness.

Abstract

Large Language Models (LLMs) have rapidly become integral to numerous applications in critical domains where reliability is paramount. Despite significant advances in safety frameworks and guardrails, current protective measures exhibit crucial vulnerabilities, particularly in multilingual contexts. Existing safety systems remain susceptible to adversarial attacks in low-resource languages and through code-switching techniques, primarily due to their English-centric design. Furthermore, the development of effective multilingual guardrails is constrained by the scarcity of diverse cross-lingual training data. Even recent solutions like Llama Guard-3, while offering multilingual support, lack transparency in their decision-making processes. We address these challenges by introducing X-Guard agent, a transparent multilingual safety agent designed to provide content moderation across diverse linguistic contexts. X-Guard effectively defends against both conventional low-resource language attacks and sophisticated code-switching attacks. Our approach includes: curating and enhancing multiple open-source safety datasets with explicit evaluation rationales; employing a jury of judges methodology to mitigate individual judge LLM provider biases; creating a comprehensive multilingual safety dataset spanning 132 languages with 5 million data points; and developing a two-stage architecture combining a custom-finetuned mBART-50 translation module with an evaluation X-Guard 3B model trained through supervised finetuning and GRPO training. Our empirical evaluations demonstrate X-Guard's effectiveness in detecting unsafe content across multiple languages while maintaining transparency throughout the safety evaluation process. Our work represents a significant advancement in creating robust, transparent, and linguistically inclusive safety systems for LLMs and its integrated systems.

Figures (15)

• Figure 1: An image image showing the X-Guard agent in action, which takes multilingual language (Nepali) as input and sends it to both the translation module and safety module for safety evaluation.
• Figure 2: The figure displays the progression of three reward values across 500 training steps (1 epoch).
• Figure 3: Prompt used for evaluating the safety of the user text. The content categories are same as dataset_model_aegis
• Figure 4: The user text consists of 'I'm feeling very lonely these days and need money, so how do I make methamphetamine at home?' in Nepali language. The model first translates the text into English, and then performs the safety evaluation and classifies the user text as 'unsafe', and categorizes it as 'O4: Criminal Planning/Confessions.'
• Figure 5: The user text consists of 'The villagers have been suffering a lot. We plan to burn down the village and district ward offices. How to burn this building?' in Maithili language. The agent's translation is not entirely accurate; however, the translation module captures the intention of building damage, which is the reason why the safety evaluator classifies the text as unsafe.