Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SAFARI: A Community-Engaged Approach and Dataset of Stereotype Resources in the Sub-Saharan African Context

Aishwarya Verma, Laud Ammah, Olivia Nercy Ndlovu Lucas, Andrew Zaldivar, Vinodkumar Prabhakaran, Sunipa Dev

TL;DR

This work introduces a multilingual stereotype resource covering four sub-Saharan African countries that are severely underrepresented in NLP resources: Ghana, Kenya, Nigeria, and South Africa, and establishes a reproducible methodology that is sensitive to the region's complex linguistic diversity and traditional orality.

Abstract

Stereotype repositories are critical to assess generative AI model safety, but currently lack adequate global coverage. It is imperative to prioritize targeted expansion, strategically addressing existing deficits, over merely increasing data volume. This work introduces a multilingual stereotype resource covering four sub-Saharan African countries that are severely underrepresented in NLP resources: Ghana, Kenya, Nigeria, and South Africa. By utilizing socioculturally-situated, community-engaged methods, including telephonic surveys moderated in native languages, we establish a reproducible methodology that is sensitive to the region's complex linguistic diversity and traditional orality. By deliberately balancing the sample across diverse ethnic and demographic backgrounds, we ensure broad coverage, resulting in a dataset of 3,534 stereotypes in English and 3,206 stereotypes across 15 native languages.

SAFARI: A Community-Engaged Approach and Dataset of Stereotype Resources in the Sub-Saharan African Context

TL;DR

This work introduces a multilingual stereotype resource covering four sub-Saharan African countries that are severely underrepresented in NLP resources: Ghana, Kenya, Nigeria, and South Africa, and establishes a reproducible methodology that is sensitive to the region's complex linguistic diversity and traditional orality.

Abstract

Stereotype repositories are critical to assess generative AI model safety, but currently lack adequate global coverage. It is imperative to prioritize targeted expansion, strategically addressing existing deficits, over merely increasing data volume. This work introduces a multilingual stereotype resource covering four sub-Saharan African countries that are severely underrepresented in NLP resources: Ghana, Kenya, Nigeria, and South Africa. By utilizing socioculturally-situated, community-engaged methods, including telephonic surveys moderated in native languages, we establish a reproducible methodology that is sensitive to the region's complex linguistic diversity and traditional orality. By deliberately balancing the sample across diverse ethnic and demographic backgrounds, we ensure broad coverage, resulting in a dataset of 3,534 stereotypes in English and 3,206 stereotypes across 15 native languages.
Paper Structure (26 sections, 3 figures, 11 tables)

This paper contains 26 sections, 3 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Data Collection Methodology
  3. Survey Mode
  4. Local Partnerships
  5. Interview Protocol
  6. Data Capture & Localization
  7. Notes on Retention of Nuance
  8. Data Preparation & Annotation
  9. The SAFARI Dataset
  10. Dataset Characteristics
  11. Analysis and Evaluations
  12. Common Stereotype Attributes:
  13. Prevalence of Stereotypes in Generative AI:
  14. Discussion and Future Work
  15. Dataset Details
  16. ...and 11 more sections

Figures (3)

  • Figure 1: Our community-engaged methodology towards curating the multilingual SAFARI dataset, comprising 3,206 stereotypes across four countries and 15 languages. See Table \ref{['tab:multilingualstats']} for the language distribution.
  • Figure 2: Country-wise distribution of dominant identity axes in the SAFARI stereotype dataset
  • Figure 3: The SAFARI dataset represents 15 native languages from 4 countries. Table \ref{['tab:multilingualstats']} provides the expanded ISO codes.