Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Does Generative AI speak Nigerian-Pidgin?: Issues about Representativeness and Bias for Multilingualism in LLMs

David Ifeoluwa Adelani, A. Seza Doğruöz, Iyanuoluwa Shode, Anuoluwapo Aremu

TL;DR

This paper interrogates the representativeness of West African Pidgin English (WAPE) and Nigerian-Pidgin (Naija) in Generative AI. It introduces the Warri benchmark to compare WAPE and Naija across domains (BBC news and Wikipedia) and conducts statistical, zero-shot transfer, and prompting experiments with MT models and large language models to assess linguistic bias. The findings show a clear bias toward WAPE in current AI systems, with Naija being underrepresented and harder to teach with limited data. Qualitative interviews with Naija Wikipedia contributors corroborate distinct writing conventions and the need for more diverse, domain-representative data. The work highlights the urgency of inclusive multilingual AI, especially for high-speaker, low-resource pidgins, and provides the Warri dataset for reproducible evaluation.

Abstract

Nigeria is a multilingual country with 500+ languages. Naija is a Nigerian Pidgin spoken by approximately 120M speakers and it is a mixed language (e.g., English, Portuguese, Yoruba, Hausa and Igbo). Although it has mainly been a spoken language until recently, there are some online platforms (e.g., Wikipedia), publishing in written Naija as well. West African Pidgin English (WAPE) is also spoken in Nigeria and it is used by BBC to broadcast news on the internet to a wider audience not only in Nigeria but also in other West African countries (e.g., Cameroon and Ghana). Through statistical analyses and Machine Translation experiments, our paper shows that these two pidgin varieties do not represent each other (i.e., there are linguistic differences in word order and vocabulary) and Generative AI operates only based on WAPE. In other words, Naija is underrepresented in Generative AI, and it is hard to teach LLMs with few examples. In addition to the statistical analyses, we also provide historical information on both pidgins as well as insights from the interviews conducted with volunteer Wikipedia contributors in Naija.

Does Generative AI speak Nigerian-Pidgin?: Issues about Representativeness and Bias for Multilingualism in LLMs

TL;DR

This paper interrogates the representativeness of West African Pidgin English (WAPE) and Nigerian-Pidgin (Naija) in Generative AI. It introduces the Warri benchmark to compare WAPE and Naija across domains (BBC news and Wikipedia) and conducts statistical, zero-shot transfer, and prompting experiments with MT models and large language models to assess linguistic bias. The findings show a clear bias toward WAPE in current AI systems, with Naija being underrepresented and harder to teach with limited data. Qualitative interviews with Naija Wikipedia contributors corroborate distinct writing conventions and the need for more diverse, domain-representative data. The work highlights the urgency of inclusive multilingual AI, especially for high-speaker, low-resource pidgins, and provides the Warri dataset for reproducible evaluation.

Abstract

Nigeria is a multilingual country with 500+ languages. Naija is a Nigerian Pidgin spoken by approximately 120M speakers and it is a mixed language (e.g., English, Portuguese, Yoruba, Hausa and Igbo). Although it has mainly been a spoken language until recently, there are some online platforms (e.g., Wikipedia), publishing in written Naija as well. West African Pidgin English (WAPE) is also spoken in Nigeria and it is used by BBC to broadcast news on the internet to a wider audience not only in Nigeria but also in other West African countries (e.g., Cameroon and Ghana). Through statistical analyses and Machine Translation experiments, our paper shows that these two pidgin varieties do not represent each other (i.e., there are linguistic differences in word order and vocabulary) and Generative AI operates only based on WAPE. In other words, Naija is underrepresented in Generative AI, and it is hard to teach LLMs with few examples. In addition to the statistical analyses, we also provide historical information on both pidgins as well as insights from the interviews conducted with volunteer Wikipedia contributors in Naija.
Paper Structure (21 sections, 2 figures, 6 tables)

This paper contains 21 sections, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Research
  3. Warri MT benchmark dataset for WAPE and Naija
  4. Other datasets in Naija
  5. Experimental setup
  6. Statistical analysis of the texts
  7. Cross-corpus zero-shot transfer results
  8. Prompting of LLMs
  9. Experimental Results
  10. Statistical analysis results
  11. Machine translation evaluation
  12. LLM performance on Warri Benchmark
  13. Mafand MT model and LLMs represent WAPE more
  14. Can we teach LLMs different genres with only a few examples?
  15. Qualitative interviews with Naija Wikipedia contributors
  16. ...and 6 more sections

Figures (2)

  • Figure 1: WAPE Locations: West African countries whose official language is English (The Gambia, Sierra Leone, Liberia, Ghana and Nigeria), Cameroon (North-Western and South-Western Anglophone region), and Krios immigrants to Equatorial Guinea. Map is obtained from kofi_wape.
  • Figure 2: Cross-corpus transfer results: Evaluation based on ChrF++