Is Lying Only Sinful in Islam? Exploring Religious Bias in Multilingual Large Language Models Across Major Religions

TL;DR

The study investigates religious bias in multilingual LLMs using the BRAND dataset spanning Islam, Hinduism, Christianity, and Buddhism in English and Bengali. By applying three prompt types across five models, the authors reveal language-dependent biases, notably Islam dominance in Bengali and varying biases in English for general norms. They demonstrate that bias interacts with language, model architecture, and norm type, highlighting risks of misrepresentation in low-resource languages and the need for robust, inclusive bias mitigation. The work also proposes HCI-centered design and community-engaged strategies to improve transparency, accountability, and fairness in AI systems addressing sensitive religious content.

Abstract

While recent developments in large language models have improved bias detection and classification, sensitive subjects like religion still present challenges because even minor errors can result in severe misunderstandings. In particular, multilingual models often misrepresent religions and have difficulties being accurate in religious contexts. To address this, we introduce BRAND: Bilingual Religious Accountable Norm Dataset, which focuses on the four main religions of South Asia: Buddhism, Christianity, Hinduism, and Islam, containing over 2,400 entries, and we used three different types of prompts in both English and Bengali. Our results indicate that models perform better in English than in Bengali and consistently display bias toward Islam, even when answering religion-neutral questions. These findings highlight persistent bias in multilingual models when similar questions are asked in different languages. We further connect our findings to the broader issues in HCI regarding religion and spirituality.

Figures (4)

• Figure 1: Illustration of model evaluation with religious norms. Both figures use the same prompts in English and Bengali. The left side shows model responses to general religious norms, while the right side shows model predictions for religion-specific norms, where models provide different answers depending on the language.
• Figure 2: Cross-religious misclassification bias in LLMs, by language. Stacked bars show, for each true religion (Islam, Hinduism, Christianity, and Buddhism), the percentage breakdown of incorrect model predictions attributed to the other three religions. Figure \ref{['fig:religion_accuracy_bn12']} shows Bengali results, and Figure \ref{['fig:religion_accuracy_en12']} shows English results.
• Figure 3: Label misclassification patterns in LLMs. For each true label category (Normal, Expected, Taboo), stacked bars depict the percentage distribution of incorrect predictions assigned to the other two labels. Figure \ref{['fig:label_bn12']} shows the Bengali dataset and Figure \ref{['fig:label_en12']} shows the English dataset.
• Figure 4: Error distribution of label misclassifications by religion. For each true religion---Islam, Hinduism, Christianity, and Buddhism--- stacked bars show how misclassified religious norms are reassigned to incorrect label categories (Normal, Expected, and Taboo). Figure \ref{['fig:religion_bn23']} presents results on the Bengali dataset, highlighting a dominant shift toward the Normal label, while Figure \ref{['fig:religion_en23']} shows the English dataset errors, which predominantly shift toward Expected, illustrating systematic cross-linguistic differences in label bias.