Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Low-Resource Neural Machine Translation Using Recurrent Neural Networks and Transfer Learning: A Case Study on English-to-Igbo

Ocheme Anthony Ekle, Biswarup Das

TL;DR

This work tackles English–Igbo translation under low-resource conditions by combining RNN-based NMT with attention and transfer learning. It demonstrates that an LSTM encoder–decoder with dot-product attention, trained with teacher forcing and tuned hyperparameters, can achieve BLEU scores near 0.38 on a ~12k parallel corpus, approaching established benchmarks. Transfer learning using MarianNMT within SimpleTransformers further boosts performance to about BLEU 0.43, surpassing public baselines and yielding roughly 70% semantic accuracy on diverse samples. Additionally, evaluating on English–French shows the model generalizes to related language pairs, underscoring the approach’s robustness; the results highlight the practical impact of integrating classical RNNs with transformer-based transfer learning for low-resource MT.

Abstract

In this study, we develop Neural Machine Translation (NMT) and Transformer-based transfer learning models for English-to-Igbo translation - a low-resource African language spoken by over 40 million people across Nigeria and West Africa. Our models are trained on a curated and benchmarked dataset compiled from Bible corpora, local news, Wikipedia articles, and Common Crawl, all verified by native language experts. We leverage Recurrent Neural Network (RNN) architectures, including Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU), enhanced with attention mechanisms to improve translation accuracy. To further enhance performance, we apply transfer learning using MarianNMT pre-trained models within the SimpleTransformers framework. Our RNN-based system achieves competitive results, closely matching existing English-Igbo benchmarks. With transfer learning, we observe a performance gain of +4.83 BLEU points, reaching an estimated translation accuracy of 70%. These findings highlight the effectiveness of combining RNNs with transfer learning to address the performance gap in low-resource language translation tasks.

Low-Resource Neural Machine Translation Using Recurrent Neural Networks and Transfer Learning: A Case Study on English-to-Igbo

TL;DR

This work tackles English–Igbo translation under low-resource conditions by combining RNN-based NMT with attention and transfer learning. It demonstrates that an LSTM encoder–decoder with dot-product attention, trained with teacher forcing and tuned hyperparameters, can achieve BLEU scores near 0.38 on a ~12k parallel corpus, approaching established benchmarks. Transfer learning using MarianNMT within SimpleTransformers further boosts performance to about BLEU 0.43, surpassing public baselines and yielding roughly 70% semantic accuracy on diverse samples. Additionally, evaluating on English–French shows the model generalizes to related language pairs, underscoring the approach’s robustness; the results highlight the practical impact of integrating classical RNNs with transformer-based transfer learning for low-resource MT.

Abstract

In this study, we develop Neural Machine Translation (NMT) and Transformer-based transfer learning models for English-to-Igbo translation - a low-resource African language spoken by over 40 million people across Nigeria and West Africa. Our models are trained on a curated and benchmarked dataset compiled from Bible corpora, local news, Wikipedia articles, and Common Crawl, all verified by native language experts. We leverage Recurrent Neural Network (RNN) architectures, including Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU), enhanced with attention mechanisms to improve translation accuracy. To further enhance performance, we apply transfer learning using MarianNMT pre-trained models within the SimpleTransformers framework. Our RNN-based system achieves competitive results, closely matching existing English-Igbo benchmarks. With transfer learning, we observe a performance gain of +4.83 BLEU points, reaching an estimated translation accuracy of 70%. These findings highlight the effectiveness of combining RNNs with transfer learning to address the performance gap in low-resource language translation tasks.

Paper Structure

This paper contains 48 sections, 5 equations, 13 figures, 14 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Natural Language Processing and Language Levels
  4. Word Vector Representation
  5. Neural Machine Translation
  6. Graph Neural Networks and Emerging Trends in NLP
  7. Background and RNN Architectures
  8. Traditional RNNs and the Vanishing Gradient Problem
  9. Long Short-Term Memory (LSTM)
  10. Gated Recurrent Unit (GRU)
  11. Summary and Implementation
  12. PROPOSED METHOD
  13. Data Collection
  14. Text Preprocessing and Analysis
  15. Data Loading and Sampling
  16. ...and 33 more sections

Figures (13)

  • Figure 1: Vanilla RNN architecture. The weight matrices $U$, $V$, and $W$ are shared across time steps to process temporal sequences.
  • Figure 2: Sample English-Igbo Dataset Visualization
  • Figure 3: Histogram of Sentence Lengths
  • Figure 4: Violin Plot of Sentence Lengths
  • Figure 5: RNN Encoder-Decoder Model with Attention
  • ...and 8 more figures