FanarGuard: A Culturally-Aware Moderation Filter for Arabic Language Models

TL;DR

FanarGuard addresses the challenge of culturally aware moderation by jointly modeling safety and cultural alignment for Arabic and English. It combines a large, judge-annotated dataset with a two-dimensional regression framework and multiple model backbones to produce compact yet effective filters. The paper introduces an Arabic cultural-safety benchmark and demonstrates that cultural alignment can be learned with small to mid-size models while maintaining strong safety performance, underscoring the importance of culture-aware safeguards for multilingual LLM deployments. The work provides an extensible, open-source framework and data pipeline bridging culture, safety, and multilingual moderation with practical deployment benefits.

Abstract

Content moderation filters are a critical safeguard against alignment failures in language models. Yet most existing filters focus narrowly on general safety and overlook cultural context. In this work, we introduce FanarGuard, a bilingual moderation filter that evaluates both safety and cultural alignment in Arabic and English. We construct a dataset of over 468K prompt and response pairs, drawn from synthetic and public datasets, scored by a panel of LLM judges on harmlessness and cultural awareness, and use it to train two filter variants. To rigorously evaluate cultural alignment, we further develop the first benchmark targeting Arabic cultural contexts, comprising over 1k norm-sensitive prompts with LLM-generated responses annotated by human raters. Results show that FanarGuard achieves stronger agreement with human annotations than inter-annotator reliability, while matching the performance of state-of-the-art filters on safety benchmarks. These findings highlight the importance of integrating cultural awareness into moderation and establish FanarGuard as a practical step toward more context-sensitive safeguards.

Figures (4)

• Figure 1: FanarGuard assesses both general safety and cultural alignment across English and Arabic languages. For example, given the prompt "Is it OK to eat frog meat?", a model aligned with Western cultural norms might answer: "Yes - **frog meat is generally safe to eat** and is considered a delicacy...” While such a response may be considered safe under general moderation standards, it conflicts with dietary norms in many Arabic-speaking societies.
• Figure 2: Overview of the data generation pipeline used to train FanarGuard, a two-dimensional moderation filter. Safety-related prompts and responses are sourced from public instruction and preference datasets (left), while culture-sensitive examples are synthetically generated using LLMs prompted on culturally controversial topics (center). All examples are then scored by a panel of LLM judges using task-specific rubrics to produce labels along two axes: harmful–harmless and aligned–unaligned (right). Each sample is scored in either Arabic or English, with the resulting label propagated to its counterpart in the other language.
• Figure 3: Distribution of safety scores in our Safety and Cultural Alignment datasets (a) before and (b) after subsampling.
• Figure 4: Distribution of cultural-sensitivity scores in our Safety and Cultural Alignment datasets (a) before and (b) after subsampling.