A fully automated and scalable Parallel Data Augmentation for Low Resource Languages using Image and Text Analytics

TL;DR

This work tackles data scarcity in low-resource languages by introducing a fully automated, image-and-text analytics pipeline to generate large-scale bilingual parallel corpora from newspaper articles. It uses images as pivots to map articles across languages, extracts and aligns text with OCR ensembles, and employs language-agnostic sentence embeddings for cross-language sentence alignment. The Konkani-Marathi corpus (and a Punjabi-Hindi extension) enable downstream MT improvements, with a BLEU increase of approximately 3 points after fine-tuning mT5, validating the quality and utility of the generated data. The approach demonstrates language-agnostic applicability and scalability, with potential to broaden NLP access to semi-literate populations by augmenting digital resources without manual annotation.

Abstract

Linguistic diversity across the world creates a disparity with the availability of good quality digital language resources thereby restricting the technological benefits to majority of human population. The lack or absence of data resources makes it difficult to perform NLP tasks for low-resource languages. This paper presents a novel scalable and fully automated methodology to extract bilingual parallel corpora from newspaper articles using image and text analytics. We validate our approach by building parallel data corpus for two different language combinations and demonstrate the value of this dataset through a downstream task of machine translation and improve over the current baseline by close to 3 BLEU points.

Figures (4)

• Figure 1: Article mapping using images as pivots.
• Figure 2: Proposed data augmentation pipeline
• Figure 3: Article sub parts and nomenclature
• Figure 4: Proposed model for our evaluation