Mitigating Social Bias in English and Urdu Language Models Using PRM-Guided Candidate Selection and Sequential Refinement

TL;DR

This work tackles social bias in English and Urdu language model outputs by proposing an inference-time framework that uses a Preference Ranking Model (PRM) to score and refine candidate completions without retraining. It compares Baseline, PRM-Select, and PRM-Sequential across 200 English prompts and 200 Urdu translations, using GPT-3.5 as the generator and GPT-4o-mini as the PRM scorer. The results show substantial bias reduction in both languages, with Urdu still lagging due to structural data limitations; PRM-Select offers robust cross-lingual fairness while PRM-Sequential can maximize fairness at the cost of Urdu utility. The work contributes an extensible evaluation framework, interpretable metrics, and insights for fairness in low-resource multilingual settings.

Abstract

Large language models (LLMs) increasingly mediate human communication, decision support, content creation, and information retrieval. Despite impressive fluency, these systems frequently produce biased or stereotypical content, especially when prompted with socially sensitive language. A growing body of research has demonstrated that such biases disproportionately affect low-resource languages, where training data is limited and culturally unrepresentative. This paper presents a comprehensive study of inference-time bias mitigation, a strategy that avoids retraining or fine-tuning and instead operates directly on model outputs. Building on preference-ranking models (PRMs), we introduce a unified evaluation framework comparing three methods: (1) baseline single-word generation, (2) PRM-Select best-of-N sampling, and (3) PRM-Sequential refinement guided by PRM critiques. We evaluate these techniques across 200 English prompts and their Urdu counterparts, designed to reflect socio-cultural contexts relevant to gender, ethnicity, religion, nationality, disability, profession, age, and socioeconomic categories. Using GPT-3.5 as a candidate generator and GPT-4o-mini as a PRM-based bias and utility scorer, we provide an extensive quantitative analysis of bias reduction, utility preservation, and cross-lingual disparities. Our findings show: (a) substantial gains over the baseline for both languages; (b) consistently lower fairness scores for Urdu across all methods, highlighting structural inequities in multilingual LLM training; and (c) distinct improvement trajectories between PRM-Select and PRM-Sequential. The study contributes an extensible methodology, interpretable metrics, and cross-lingual comparisons that can support future work on fairness evaluation in low-resource languages.

Figures (6)

• Figure 1: System diagram of the inference-time debiasing pipeline used in this study.
• Figure 2: Bias Across Methods (English vs. Urdu). Higher scores indicate lower stereotype presence.
• Figure 3: Utility Across Methods (English vs. Urdu). Higher scores indicate better semantic fit.
• Figure 4: Composite Score Across Methods.
• Figure 5: Heatmap of English–Urdu differences for each method.