Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mafoko: Structuring and Building Open Multilingual Terminologies for South African NLP

Vukosi Marivate, Isheanesu Dzingirai, Fiskani Banda, Richard Lastrucci, Thapelo Sindane, Keabetswe Madumo, Kayode Olaleye, Abiodun Modupe, Unarine Netshifhefhe, Herkulaas Combrink, Mohlatlego Nakeng, Matome Ledwaba

TL;DR

The paper tackles the lack of structured, machine-readable terminologies for South Africa's official languages by introducing Mafoko, a framework that aggregates, digitises, and standardises government and academic assets into open datasets licensed under the Africa-centered NOODL. It releases Mafoko v0 and demonstrates immediate utility by integrating the terminologies into a Retrieval-Augmented Generation pipeline to improve English-to-Tshivenda translation across Mathematics and Election domains. The work provides a scalable, equitable foundation for developing robust multilingual NLP technologies that reflect South Africa's linguistic diversity, supported by open data platforms and community governance. This approach has practical implications for model evaluation, linguistic research, and the development of inclusive language technologies in Africa.

Abstract

The critical lack of structured terminological data for South Africa's official languages hampers progress in multilingual NLP, despite the existence of numerous government and academic terminology lists. These valuable assets remain fragmented and locked in non-machine-readable formats, rendering them unusable for computational research and development. Mafoko addresses this challenge by systematically aggregating, cleaning, and standardising these scattered resources into open, interoperable datasets. We introduce the foundational Mafoko dataset, released under the equitable, Africa-centered NOODL framework. To demonstrate its immediate utility, we integrate the terminology into a Retrieval-Augmented Generation (RAG) pipeline. Experiments show substantial improvements in the accuracy and domain-specific consistency of English-to-Tshivenda machine translation for large language models. Mafoko provides a scalable foundation for developing robust and equitable NLP technologies, ensuring South Africa's rich linguistic diversity is represented in the digital age.

Mafoko: Structuring and Building Open Multilingual Terminologies for South African NLP

TL;DR

The paper tackles the lack of structured, machine-readable terminologies for South Africa's official languages by introducing Mafoko, a framework that aggregates, digitises, and standardises government and academic assets into open datasets licensed under the Africa-centered NOODL. It releases Mafoko v0 and demonstrates immediate utility by integrating the terminologies into a Retrieval-Augmented Generation pipeline to improve English-to-Tshivenda translation across Mathematics and Election domains. The work provides a scalable, equitable foundation for developing robust multilingual NLP technologies that reflect South Africa's linguistic diversity, supported by open data platforms and community governance. This approach has practical implications for model evaluation, linguistic research, and the development of inclusive language technologies in Africa.

Abstract

The critical lack of structured terminological data for South Africa's official languages hampers progress in multilingual NLP, despite the existence of numerous government and academic terminology lists. These valuable assets remain fragmented and locked in non-machine-readable formats, rendering them unusable for computational research and development. Mafoko addresses this challenge by systematically aggregating, cleaning, and standardising these scattered resources into open, interoperable datasets. We introduce the foundational Mafoko dataset, released under the equitable, Africa-centered NOODL framework. To demonstrate its immediate utility, we integrate the terminology into a Retrieval-Augmented Generation (RAG) pipeline. Experiments show substantial improvements in the accuracy and domain-specific consistency of English-to-Tshivenda machine translation for large language models. Mafoko provides a scalable foundation for developing robust and equitable NLP technologies, ensuring South Africa's rich linguistic diversity is represented in the digital age.

Paper Structure

This paper contains 23 sections, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Motivation
  3. Methodology
  4. Source Identification
  5. Domain Coverage
  6. Challenges in Data Acquisition
  7. Data Curation and Structuring
  8. Data Release and Availability
  9. Licensing under NOODL
  10. Terminology Applications
  11. A Benchmark for Multilingual NLP Evaluation
  12. A Resource for Linguistic and Sociolinguistic Inquiry
  13. Improving Translations with Terminology lists and RAG
  14. Task and Models
  15. Experimental Conditions
  16. ...and 8 more sections

Figures (3)

  • Figure 1: Formatting differences across different terminology lists.
  • Figure 2: Overview of RAG and LLM Pipelines
  • Figure 3: Translation performance metrics comparison of GPT-4o-mini and LLaMA3-8B models on English to Tshivenda Mathematics and Election datasets.