Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cascading Adaptors to Leverage English Data to Improve Performance of Question Answering for Low-Resource Languages

Hariom A. Pandya, Bhavik Ardeshna, Brijesh S. Bhatt

TL;DR

It is observed that stacking the language and the task adapters improves the multilingual transformer models’ performance significantly for low-resource languages.

Abstract

Transformer based architectures have shown notable results on many down streaming tasks including question answering. The availability of data, on the other hand, impedes obtaining legitimate performance for low-resource languages. In this paper, we investigate the applicability of pre-trained multilingual models to improve the performance of question answering in low-resource languages. We tested four combinations of language and task adapters using multilingual transformer architectures on seven languages similar to MLQA dataset. Additionally, we have also proposed zero-shot transfer learning of low-resource question answering using language and task adapters. We observed that stacking the language and the task adapters improves the multilingual transformer models' performance significantly for low-resource languages.

Cascading Adaptors to Leverage English Data to Improve Performance of Question Answering for Low-Resource Languages

TL;DR

It is observed that stacking the language and the task adapters improves the multilingual transformer models’ performance significantly for low-resource languages.

Abstract

Transformer based architectures have shown notable results on many down streaming tasks including question answering. The availability of data, on the other hand, impedes obtaining legitimate performance for low-resource languages. In this paper, we investigate the applicability of pre-trained multilingual models to improve the performance of question answering in low-resource languages. We tested four combinations of language and task adapters using multilingual transformer architectures on seven languages similar to MLQA dataset. Additionally, we have also proposed zero-shot transfer learning of low-resource question answering using language and task adapters. We observed that stacking the language and the task adapters improves the multilingual transformer models' performance significantly for low-resource languages.
Paper Structure (11 sections, 2 figures, 6 tables)

This paper contains 11 sections, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Proposed Approach
  3. Cross-Lingual Tuning of Task Adapter and Language Adapters
  4. Multi-Task Adapter for Cross-Lingual Transfer
  5. Experimental setups
  6. Setup A
  7. Setup B
  8. Setup C
  9. Setup D
  10. Observations
  11. Conclusions

Figures (2)

  • Figure 1: Experimental architecture- SETUP A: mBERT and XLM-R for QA, SETUP B: XLM-R with task adapters setup, SETUP C: XLM-R with language and language + task adapter and SETUP D: MAD-X setup for XLM-R
  • Figure 2: The performance of the different heads. The Y-axis here denotes the F1 score