Harnessing Transfer Learning from Swahili: Advancing Solutions for Comorian Dialects

TL;DR

This work addresses the critical data gap for Comorian languages by employing transfer learning from Swahili to bootstrap NLP tools for Comorian dialects. It introduces dataset construction strategies that mix Swahili corpora with Comorian data based on lexical proximity, and uses pseudo-labeling to create MT and ASR resources under tight resource constraints. Powered by mT5 and Whisper, the approach yields initial MT performance with ROUGE scores around 0.68/0.42/0.65 and ASR performance with a WER of 39.50% and CER of 13.76%, demonstrating feasibility and a path forward for democratizing NLP for extremely low-resource languages. The work lays groundwork for broader adoption and preservation of Comorian linguistic heritage in the digital age, with implications for cross-lingual transfer in COVID-era low-resource NLP deployments across Africa.

Abstract

If today some African languages like Swahili have enough resources to develop high-performing Natural Language Processing (NLP) systems, many other languages spoken on the continent are still lacking such support. For these languages, still in their infancy, several possibilities exist to address this critical lack of data. Among them is Transfer Learning, which allows low-resource languages to benefit from the good representation of other languages that are similar to them. In this work, we adopt a similar approach, aiming to pioneer NLP technologies for Comorian, a group of four languages or dialects belonging to the Bantu family. Our approach is initially motivated by the hypothesis that if a human can understand a different language from their native language with little or no effort, it would be entirely possible to model this process on a machine. To achieve this, we consider ways to construct Comorian datasets mixed with Swahili. One thing to note here is that in terms of Swahili data, we only focus on elements that are closest to Comorian by calculating lexical distances between candidate and source data. We empirically test this hypothesis in two use cases: Automatic Speech Recognition (ASR) and Machine Translation (MT). Our MT model achieved ROUGE-1, ROUGE-2, and ROUGE-L scores of 0.6826, 0.42, and 0.6532, respectively, while our ASR system recorded a WER of 39.50\% and a CER of 13.76\%. This research is crucial for advancing NLP in underrepresented languages, with potential to preserve and promote Comorian linguistic heritage in the digital age.

Figures (2)

• Figure 1: Approximate locations of the sixteen Guthrie Bantu zones.
• Figure 2: NMT Data Construction.