Myanmar XNLI: Building a Dataset and Exploring Low-resource Approaches to Natural Language Inference with Myanmar
Aung Kyaw Htet, Mark Dras
TL;DR
Myanmar XNLI extends the Cross-lingual Natural Language Inference benchmark to a low-resource language, Myanmar, by constructing myXNLI via a two-stage crowd-sourced workflow with expert revision. The study establishes baseline performance for multilingual models (XLM-R, mDeBERTa) and monolingual Myanmar models, and investigates data augmentation strategies—adversarial, multilingual, cross-matched, and genre‑based side inputs—that yield up to about 2 percentage points gains. It also demonstrates cross-lingual transfer from English and evaluates translate-test/train setups to provide practical baselines when task-specific multilingual resources are scarce. Moreover, the work analyzes translation quality, semantic consistency after translation, and the generalizability of augmentation methods to other low-resource languages (e.g., Swahili, Urdu). The findings highlight the importance of data quality, cross-lingual strategies, and metadata augmentation for advancing NLI in low-resource languages and offer a scalable blueprint for similar efforts in other languages.
Abstract
Despite dramatic recent progress in NLP, it is still a major challenge to apply Large Language Models (LLM) to low-resource languages. This is made visible in benchmarks such as Cross-Lingual Natural Language Inference (XNLI), a key task that demonstrates cross-lingual capabilities of NLP systems across a set of 15 languages. In this paper, we extend the XNLI task for one additional low-resource language, Myanmar, as a proxy challenge for broader low-resource languages, and make three core contributions. First, we build a dataset called Myanmar XNLI (myXNLI) using community crowd-sourced methods, as an extension to the existing XNLI corpus. This involves a two-stage process of community-based construction followed by expert verification; through an analysis, we demonstrate and quantify the value of the expert verification stage in the context of community-based construction for low-resource languages. We make the myXNLI dataset available to the community for future research. Second, we carry out evaluations of recent multilingual language models on the myXNLI benchmark, as well as explore data-augmentation methods to improve model performance. Our data-augmentation methods improve model accuracy by up to 2 percentage points for Myanmar, while uplifting other languages at the same time. Third, we investigate how well these data-augmentation methods generalise to other low-resource languages in the XNLI dataset.