Datasets for Multilingual Answer Sentence Selection

TL;DR

AS2 research has been English-centric, limiting QA deployment across languages. The authors create three large multilingual AS2 corpora (mASNQ, mWikiQA, mTREC-QA) by translating ASNQ, WikiQA, and TREC-QA into five European languages using a state-of-the-art MT model. They validate the approach with Transformer-based rankers (XLM-RoBERTa, mDeBERTa) using a two-stage TANDA training (transfer on English AS2 corpora, adaptation on translated datasets) and demonstrate competitive performance with English baselines, including robust zero-shot transfer. The work significantly lowers the resource barrier for multilingual QA and provides high-quality data and baselines for future research.

Abstract

Answer Sentence Selection (AS2) is a critical task for designing effective retrieval-based Question Answering (QA) systems. Most advancements in AS2 focus on English due to the scarcity of annotated datasets for other languages. This lack of resources prevents the training of effective AS2 models in different languages, creating a performance gap between QA systems in English and other locales. In this paper, we introduce new high-quality datasets for AS2 in five European languages (French, German, Italian, Portuguese, and Spanish), obtained through supervised Automatic Machine Translation (AMT) of existing English AS2 datasets such as ASNQ, WikiQA, and TREC-QA using a Large Language Model (LLM). We evaluated our approach and the quality of the translated datasets through multiple experiments with different Transformer architectures. The results indicate that our datasets are pivotal in producing robust and powerful multilingual AS2 models, significantly contributing to closing the performance gap between English and other languages.