GeniL: A Multilingual Dataset on Generalizing Language

TL;DR

The paper tackles the problem that co-occurrence-based stereotype detection overlooks linguistic context and cross-linguistic variation. It introduces GeniL, a multilingual dataset of over 50K sentences in 9 languages, annotated for generalizing language and distinguishing promoting versus mentioning of generalizations. Analyses reveal that generalization is infrequent and highly language- and identity-dependent, implying that simple co-occurrence metrics are biased. The authors train multilingual classifiers (mT5-XXL and PaLM-2 S) under diverse settings, achieving an overall PR-AUC of 58.7 and showing that multilingual training improves performance across languages. This work provides data and tools for nuanced stereotype evaluation and safer language technologies, while outlining limitations and directions for future multilingual expansion and discourse-level generalization modeling.

Abstract

Generative language models are transforming our digital ecosystem, but they often inherit societal biases, for instance stereotypes associating certain attributes with specific identity groups. While whether and how these biases are mitigated may depend on the specific use cases, being able to effectively detect instances of stereotype perpetuation is a crucial first step. Current methods to assess presence of stereotypes in generated language rely on simple template or co-occurrence based measures, without accounting for the variety of sentential contexts they manifest in. We argue that understanding the sentential context is crucial for detecting instances of generalization. We distinguish two types of generalizations: (1) language that merely mentions the presence of a generalization ("people think the French are very rude"), and (2) language that reinforces such a generalization ("as French they must be rude"), from non-generalizing context ("My French friends think I am rude"). For meaningful stereotype evaluations, we need to reliably distinguish such instances of generalizations. We introduce the new task of detecting generalization in language, and build GeniL, a multilingual dataset of over 50K sentences from 9 languages (English, Arabic, Bengali, Spanish, French, Hindi, Indonesian, Malay, and Portuguese) annotated for instances of generalizations. We demonstrate that the likelihood of a co-occurrence being an instance of generalization is usually low, and varies across different languages, identity groups, and attributes. We build classifiers to detect generalization in language with an overall PR-AUC of 58.7, with varying degrees of performance across languages. Our research provides data and tools to enable a nuanced understanding of stereotype perpetuation, a crucial step towards more inclusive and responsible language technologies.

Figures (5)

• Figure 1: The process of creating GeniL dataset; we used stereotypical associations (pairs of identity term and attribute) from the Multilingual SeeGULL bhutani2024seegull, and query the mC4 dataset to collect sentences which mention those pairs. During a data annotation process with trained annotators we collected two labels for each sentence: (1) whether the sentence is generalizing, and if so (2) is it promoting a generalization or mentioning.
• Figure 2: The frequency with which associations and identities appear in generalizing contexts varies across languages. Each color-coded point represent an association or identity and the black dot and lines respectively represent the means and standard deviations. The high deviations from the average suggests that relying solely on the co-occurrence of stereotypical associations may result in differential misclassification rates for different associations and identity terms.
• Figure 3: Emergence of associations in generalizing and not generalizing language. (a) shows the likelihood of stereotypical associations appearing in different contexts, in almost all languages this ratio in smaller than 10%. (b) focuses on stereotypical associations that are marked as offensive. Offensive stereotypical associations are extremely unlikely to appear in generalizing language.
• Figure 4: The results of multilingual classifiers on three tasks: (1) is the generalizing language Promoting or Mentioning an association, and what are the (2) identity term, and (3) attribute that are shaping the association. Models are tested on different languages and the performance (calculated as the F1-score) shows that using the multilingual GeniL leads to best performance in almost all tasks and languages.
• Figure 5: The frequency with which associations and identities appear in generalizing contexts varies across languages. This is the same info as Figure \ref{['fig:identity']} using a box plot