Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

IndicSQuAD: A Comprehensive Multilingual Question Answering Dataset for Indic Languages

Sharvi Endait, Ruturaj Ghatage, Aditya Kulkarni, Rajlaxmi Patil, Raviraj Joshi

TL;DR

IndicSQuAD tackles the paucity of QA resources for Indic languages by translating and aligning SQuAD 2.0 into 10 Indic languages with a robust span-mapping pipeline inspired by MahaSQuAD. The dataset provides per-language train/validation/test splits and is evaluated with both language-specific monolingual BERTs and MuRIL-BERT, revealing that monolingual models generally outperform multilingual baselines in low-resource settings. Key contributions include a scalable translation methodology with precise answer-span alignment, a comprehensive multilingual benchmark, and publicly released models and data to foster research. The work highlights ongoing challenges in cross-language transfer and points to future directions like expanding to more languages, domain-specific QA, and multimodal data to further reduce digital inequality in the Indian context.

Abstract

The rapid progress in question-answering (QA) systems has predominantly benefited high-resource languages, leaving Indic languages largely underrepresented despite their vast native speaker base. In this paper, we present IndicSQuAD, a comprehensive multi-lingual extractive QA dataset covering nine major Indic languages, systematically derived from the SQuAD dataset. Building on previous work with MahaSQuAD for Marathi, our approach adapts and extends translation techniques to maintain high linguistic fidelity and accurate answer-span alignment across diverse languages. IndicSQuAD comprises extensive training, validation, and test sets for each language, providing a robust foundation for model development. We evaluate baseline performances using language-specific monolingual BERT models and the multilingual MuRIL-BERT. The results indicate some challenges inherent in low-resource settings. Moreover, our experiments suggest potential directions for future work, including expanding to additional languages, developing domain-specific datasets, and incorporating multimodal data. The dataset and models are publicly shared at https://github.com/l3cube-pune/indic-nlp

IndicSQuAD: A Comprehensive Multilingual Question Answering Dataset for Indic Languages

TL;DR

IndicSQuAD tackles the paucity of QA resources for Indic languages by translating and aligning SQuAD 2.0 into 10 Indic languages with a robust span-mapping pipeline inspired by MahaSQuAD. The dataset provides per-language train/validation/test splits and is evaluated with both language-specific monolingual BERTs and MuRIL-BERT, revealing that monolingual models generally outperform multilingual baselines in low-resource settings. Key contributions include a scalable translation methodology with precise answer-span alignment, a comprehensive multilingual benchmark, and publicly released models and data to foster research. The work highlights ongoing challenges in cross-language transfer and points to future directions like expanding to more languages, domain-specific QA, and multimodal data to further reduce digital inequality in the Indian context.

Abstract

The rapid progress in question-answering (QA) systems has predominantly benefited high-resource languages, leaving Indic languages largely underrepresented despite their vast native speaker base. In this paper, we present IndicSQuAD, a comprehensive multi-lingual extractive QA dataset covering nine major Indic languages, systematically derived from the SQuAD dataset. Building on previous work with MahaSQuAD for Marathi, our approach adapts and extends translation techniques to maintain high linguistic fidelity and accurate answer-span alignment across diverse languages. IndicSQuAD comprises extensive training, validation, and test sets for each language, providing a robust foundation for model development. We evaluate baseline performances using language-specific monolingual BERT models and the multilingual MuRIL-BERT. The results indicate some challenges inherent in low-resource settings. Moreover, our experiments suggest potential directions for future work, including expanding to additional languages, developing domain-specific datasets, and incorporating multimodal data. The dataset and models are publicly shared at https://github.com/l3cube-pune/indic-nlp
Paper Structure (17 sections, 1 figure, 4 tables)

This paper contains 17 sections, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Translation Based Approaches
  4. Natively annotated datasets
  5. Natively Annotated Datasets
  6. Experimental Setup
  7. Data Collection
  8. Translation Methodologies
  9. Languages covered
  10. Robust approach
  11. Dataset Statistics
  12. Benchmarking and Experiments
  13. Models used
  14. Experimental Setup
  15. Results
  16. ...and 2 more sections

Figures (1)

  • Figure 1: Algorithm for obtaining the answer and the answer span from the context