Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The IgboAPI Dataset: Empowering Igbo Language Technologies through Multi-dialectal Enrichment

Chris Chinenye Emezue, Ifeoma Okoh, Chinedu Mbonu, Chiamaka Chukwuneke, Daisy Lal, Ignatius Ezeani, Paul Rayson, Ijemma Onwuzulike, Chukwuma Okeke, Gerald Nweya, Bright Ogbonna, Chukwuebuka Oraegbunam, Esther Chidinma Awo-Ndubuisi, Akudo Amarachukwu Osuagwu, Obioha Nmezi

TL;DR

The paper addresses the endangered status of the Igbo language and the need for dialect-aware NLP resources. It introduces IgboAPI, a multi-dialect Igbo–English dictionary built by expert lexicographers, with a focus on dialect metadata and cross-task applicability. Through two studies—bootstrapping an Igbo semantic lexicon via PyMUSAS and finetuning a state-of-the-art MT model on a dialect-infused Igbo–English corpus—it demonstrates substantial improvements in dialect handling, including a BLEU gain of 55.08 on standard test sets and 51.20 on unseen dialects, as well as a +4.61 BLEU in ablation tests. The work highlights the practical impact of dialect-aware resources for Igbo NLP and provides a release under CC BY-NC-SA 4.0 DEED to encourage adoption and further development.

Abstract

The Igbo language is facing a risk of becoming endangered, as indicated by a 2025 UNESCO study. This highlights the need to develop language technologies for Igbo to foster communication, learning and preservation. To create robust, impactful, and widely adopted language technologies for Igbo, it is essential to incorporate the multi-dialectal nature of the language. The primary obstacle in achieving dialectal-aware language technologies is the lack of comprehensive dialectal datasets. In response, we present the IgboAPI dataset, a multi-dialectal Igbo-English dictionary dataset, developed with the aim of enhancing the representation of Igbo dialects. Furthermore, we illustrate the practicality of the IgboAPI dataset through two distinct studies: one focusing on Igbo semantic lexicon and the other on machine translation. In the semantic lexicon project, we successfully establish an initial Igbo semantic lexicon for the Igbo semantic tagger, while in the machine translation study, we demonstrate that by finetuning existing machine translation systems using the IgboAPI dataset, we significantly improve their ability to handle dialectal variations in sentences.

The IgboAPI Dataset: Empowering Igbo Language Technologies through Multi-dialectal Enrichment

TL;DR

The paper addresses the endangered status of the Igbo language and the need for dialect-aware NLP resources. It introduces IgboAPI, a multi-dialect Igbo–English dictionary built by expert lexicographers, with a focus on dialect metadata and cross-task applicability. Through two studies—bootstrapping an Igbo semantic lexicon via PyMUSAS and finetuning a state-of-the-art MT model on a dialect-infused Igbo–English corpus—it demonstrates substantial improvements in dialect handling, including a BLEU gain of 55.08 on standard test sets and 51.20 on unseen dialects, as well as a +4.61 BLEU in ablation tests. The work highlights the practical impact of dialect-aware resources for Igbo NLP and provides a release under CC BY-NC-SA 4.0 DEED to encourage adoption and further development.

Abstract

The Igbo language is facing a risk of becoming endangered, as indicated by a 2025 UNESCO study. This highlights the need to develop language technologies for Igbo to foster communication, learning and preservation. To create robust, impactful, and widely adopted language technologies for Igbo, it is essential to incorporate the multi-dialectal nature of the language. The primary obstacle in achieving dialectal-aware language technologies is the lack of comprehensive dialectal datasets. In response, we present the IgboAPI dataset, a multi-dialectal Igbo-English dictionary dataset, developed with the aim of enhancing the representation of Igbo dialects. Furthermore, we illustrate the practicality of the IgboAPI dataset through two distinct studies: one focusing on Igbo semantic lexicon and the other on machine translation. In the semantic lexicon project, we successfully establish an initial Igbo semantic lexicon for the Igbo semantic tagger, while in the machine translation study, we demonstrate that by finetuning existing machine translation systems using the IgboAPI dataset, we significantly improve their ability to handle dialectal variations in sentences.
Paper Structure (26 sections, 6 figures, 5 tables)

This paper contains 26 sections, 6 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. The IgboAPI Project
  4. Background
  5. Creating the IgboAPI Dataset
  6. Key Participants and Roles:
  7. Lexicography Tasks:
  8. Quality Assurance:
  9. Training:
  10. The IgboAPI dataset
  11. Experiments
  12. Igbo Semantic Lexicon
  13. Experimental Methodology
  14. Bootstrapping Process
  15. Results & Discussion
  16. ...and 11 more sections

Figures (6)

  • Figure 1: Illustrating the attributes of each word entry in the IgboAPI dataset using the example nwokē. Each sourced Igbo word contains the following attributes: 1) the headword which is the Standard Igbo variant of the word; 2) part of speech; 3) English definition of the Igbo word; 4) sentence of examples of the word (provided in both Igbo and English languages). The required number of example sentences is 1-4; 5) other words related to the given Igbo word; 6) word stems; 7) dialectal variations of the Igbo word. The dialectal variations provided depend on the lexicographers.
  • Figure 2: Dialects covered in the IgboAPI dataset and their number of words.
  • Figure 3: Histogram of the count of dialects for each word. Each word can possess one or more Igbo dialects. Many words in our dataset have more than one dialect (with some words having as high as 10 diverse dialectal variations. This highlights the multidialectal nature of the Igbo language.
  • Figure 4: Distribution of Word Classes.
  • Figure 5: Average BLEU score for each dialect in the StandardTestSet.
  • ...and 1 more figures