Cheetah: Natural Language Generation for 517 African Languages

TL;DR

Cheetah introduces a 580M-parameter encoder-decoder NLG model trained on a 42G, multi-domain corpus to cover 517 African languages, addressing critical resource gaps in African NLP. It benchmarks against existing multilingual models and introduces AfroNLG, a comprehensive six-task NLG suite spanning 527 languages, enabling robust evaluation of generation capabilities. Across six task clusters, Cheetah achieves state-of-the-art performance on many tasks, with notable improvements in translation, summarization, and title generation, complemented by a rigorous linguistic and human evaluation. The work advances Afrocentric NLP by enabling broad linguistic inclusion, promoting language preservation, and providing public access to resources while acknowledging limitations and the need for careful ethical consideration.

Abstract

Low-resource African languages pose unique challenges for natural language processing (NLP) tasks, including natural language generation (NLG). In this paper, we develop Cheetah, a massively multilingual NLG language model for African languages. Cheetah supports 517 African languages and language varieties, allowing us to address the scarcity of NLG resources and provide a solution to foster linguistic diversity. We demonstrate the effectiveness of Cheetah through comprehensive evaluations across six generation downstream tasks. In five of the six tasks, Cheetah significantly outperforms other models, showcasing its remarkable performance for generating coherent and contextually appropriate text in a wide range of African languages. We additionally conduct a detailed human evaluation to delve deeper into the linguistic capabilities of Cheetah. The introduction of Cheetah has far-reaching benefits for linguistic diversity. By leveraging pretrained models and adapting them to specific languages, our approach facilitates the development of practical NLG applications for African communities. The findings of this study contribute to advancing NLP research in low-resource settings, enabling greater accessibility and inclusion for African languages in a rapidly expanding digital landscape. We publicly release our models for research.

Figures (8)

• Figure 1: Cheetah is trained on $517$ African languages and language varieties across $14$ language families. The languages are domiciled in $50$ of $54$ African countries and are written in six different scripts.
• Figure 2: Examples from the mask-one and mask-at-least-one cloze task data.
• Figure 3: Faithfulness and fluency for Hausa, Swahili, and Yorùbá
• Figure F.1: Faithfulness and fluency for Intransitives in Hausa, Swahili, and Yorùbá
• Figure F.2: Faithfulness and fluency for Transitives in Hausa, Swahili, and Yorùbá