Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Shiksha: A Technical Domain focused Translation Dataset and Model for Indian Languages

Advait Joglekar, Srinivasan Umesh

TL;DR

The paper tackles the problem of neural machine translation for Indian languages in scientific, technical, and educational domains by constructing Shiksha, a large domain-specific parallel corpus sourced from NPTEL transcripts. It combines meticulous data extraction, regex-based cleaning, and LABSE-guided bitext mining (via SentAlign) to produce roughly 2.8 million sentence-pairs across 8 English–Indic and 28 Indic–Indic pairs. A base multilingual model (NLLB-200 3.3B) is fine-tuned with LoRA using three strategies, with the best approach achieving strong in-domain gains and better generalization on Flores+ benchmarks, approaching IndicTrans2 performance. The work also introduces Translingua, a tool enabling faster, higher-quality subtitle translations, and highlights the importance of domain-specific data for improving NMT in low-resource languages, while acknowledging limitations and the need for broader domain coverage and human evaluation.

Abstract

Neural Machine Translation (NMT) models are typically trained on datasets with limited exposure to Scientific, Technical and Educational domains. Translation models thus, in general, struggle with tasks that involve scientific understanding or technical jargon. Their performance is found to be even worse for low-resource Indian languages. Finding a translation dataset that tends to these domains in particular, poses a difficult challenge. In this paper, we address this by creating a multilingual parallel corpus containing more than 2.8 million rows of English-to-Indic and Indic-to-Indic high-quality translation pairs across 8 Indian languages. We achieve this by bitext mining human-translated transcriptions of NPTEL video lectures. We also finetune and evaluate NMT models using this corpus and surpass all other publicly available models at in-domain tasks. We also demonstrate the potential for generalizing to out-of-domain translation tasks by improving the baseline by over 2 BLEU on average for these Indian languages on the Flores+ benchmark. We are pleased to release our model and dataset via this link: https://huggingface.co/SPRINGLab.

Shiksha: A Technical Domain focused Translation Dataset and Model for Indian Languages

TL;DR

The paper tackles the problem of neural machine translation for Indian languages in scientific, technical, and educational domains by constructing Shiksha, a large domain-specific parallel corpus sourced from NPTEL transcripts. It combines meticulous data extraction, regex-based cleaning, and LABSE-guided bitext mining (via SentAlign) to produce roughly 2.8 million sentence-pairs across 8 English–Indic and 28 Indic–Indic pairs. A base multilingual model (NLLB-200 3.3B) is fine-tuned with LoRA using three strategies, with the best approach achieving strong in-domain gains and better generalization on Flores+ benchmarks, approaching IndicTrans2 performance. The work also introduces Translingua, a tool enabling faster, higher-quality subtitle translations, and highlights the importance of domain-specific data for improving NMT in low-resource languages, while acknowledging limitations and the need for broader domain coverage and human evaluation.

Abstract

Neural Machine Translation (NMT) models are typically trained on datasets with limited exposure to Scientific, Technical and Educational domains. Translation models thus, in general, struggle with tasks that involve scientific understanding or technical jargon. Their performance is found to be even worse for low-resource Indian languages. Finding a translation dataset that tends to these domains in particular, poses a difficult challenge. In this paper, we address this by creating a multilingual parallel corpus containing more than 2.8 million rows of English-to-Indic and Indic-to-Indic high-quality translation pairs across 8 Indian languages. We achieve this by bitext mining human-translated transcriptions of NPTEL video lectures. We also finetune and evaluate NMT models using this corpus and surpass all other publicly available models at in-domain tasks. We also demonstrate the potential for generalizing to out-of-domain translation tasks by improving the baseline by over 2 BLEU on average for these Indian languages on the Flores+ benchmark. We are pleased to release our model and dataset via this link: https://huggingface.co/SPRINGLab.

Paper Structure

This paper contains 19 sections, 6 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Where does Present-day MT fail?
  3. Related Work
  4. The Dataset
  5. First, the Source
  6. Data Cleaning and Extraction
  7. Bitext Mining
  8. Data Collation
  9. Data Analysis
  10. The Model
  11. Baseline Model selection
  12. Training
  13. Evaluation
  14. Translingua
  15. Conclusion
  16. ...and 4 more sections

Figures (6)

  • Figure 1: Translation Pair Counts (in thousands)
  • Figure 2: Average LABSE score across language pairs
  • Figure 3: A sample page from a bilingual document
  • Figure 4: Chrf++ comparison between NLLB, IT2 and our model across all Indian languages. The size of the bubble represents the population of the speakers.
  • Figure 5: A screenshot from the Translingua tool
  • ...and 1 more figures