An Empirical Study of Gendered Stereotypes in Emotional Attributes for Bangla in Multilingual Large Language Models

TL;DR

The study tackles gendered stereotypes in emotional attribution within Bangla by deploying a zero-shot prompting framework across three multilingual systems to elicit emotion attributes for male and female personas. It leverages the Bangla Emonoba dataset to collect thousands of prompts and analyzes both quantitative distributions and qualitative word-embedding patterns, revealing robust gendered biases such as women being tied to sadness and fear while men are linked to anger and pride, with Joy showing balance. Statistical tests ($p<0.05$ via a $\ ext{χ}^2$ test with Bonferroni correction) confirm these biases across templates and models, and semantic embedding analyses demonstrate distinct gender-specific emotion vocabularies. The work highlights practical implications for Bangla NLP, calling for de-biasing during fine-tuning and the creation of bias-benchmark frameworks for low-resource languages, while providing open data and code to support future research.

Abstract

The influence of Large Language Models (LLMs) is rapidly growing, automating more jobs over time. Assessing the fairness of LLMs is crucial due to their expanding impact. Studies reveal the reflection of societal norms and biases in LLMs, which creates a risk of propagating societal stereotypes in downstream tasks. Many studies on bias in LLMs focus on gender bias in various NLP applications. However, there's a gap in research on bias in emotional attributes, despite the close societal link between emotion and gender. This gap is even larger for low-resource languages like Bangla. Historically, women are associated with emotions like empathy, fear, and guilt, while men are linked to anger, bravado, and authority. This pattern reflects societal norms in Bangla-speaking regions. We offer the first thorough investigation of gendered emotion attribution in Bangla for both closed and open source LLMs in this work. Our aim is to elucidate the intricate societal relationship between gender and emotion specifically within the context of Bangla. We have been successful in showing the existence of gender bias in the context of emotions in Bangla through analytical methods and also show how emotion attribution changes on the basis of gendered role selection in LLMs. All of our resources including code and data are made publicly available to support future research on Bangla NLP. Warning: This paper contains explicit stereotypical statements that many may find offensive.

Figures (6)

• Figure 1: The two different persona based system instruction templates used in prompting LLMs for this study.
• Figure 2: The prompt template and an example of prompt and response. (Note that the translations are only for understanding and not used in prompting)
• Figure 3: Distributions of different emotion attributes for male and female genders for all LLMs applying two different prompt templates. The top eight attributes were only considered here. The English translation for attributes is also provided.
• Figure 4: Comparison of Most Attributed Emotion Words Between Genders (Prompt Template I2). Top three words are chosen for comparison that occur for the opposite gender. Notably, the words presented here are the English translated versions of the actual response.
• Figure 5: The Junto Wheel of Emotion