Enhancing the QA Model through a Multi-domain Debiasing Framework

TL;DR

This work addresses biases in QA systems that degrade performance under adversarial conditions. It applies a multi-domain debiasing framework combining knowledge distillation, debiasing, and domain expansion to ELECTRA-small trained on SQuAD v1.1, evaluating on AddSent and AddOneSent and extending to HotpotQA and MRQA domains. The authors identify three main failure modes—lexical bias, numerical reasoning errors, and entity recognition errors—and demonstrate improvements up to 2.6 percentage points in EM and F1 across test sets, with stronger gains in adversarial contexts when expanding domains. While benefits are evident, challenges remain in addressing entity recognition and achieving larger adversarial gains, along with higher computational costs for domain expansion. Overall, the results indicate that targeted cross-domain debiasing can enhance QA robustness and inform future bias-mitigation strategies.

Abstract

Question-answering (QA) models have advanced significantly in machine reading comprehension but often exhibit biases that hinder their performance, particularly with complex queries in adversarial conditions. This study evaluates the ELECTRA-small model on the Stanford Question Answering Dataset (SQuAD) v1.1 and adversarial datasets AddSent and AddOneSent. By identifying errors related to lexical bias, numerical reasoning, and entity recognition, we develop a multi-domain debiasing framework incorporating knowledge distillation, debiasing techniques, and domain expansion. Our results demonstrate up to 2.6 percentage point improvements in Exact Match (EM) and F1 scores across all test sets, with gains in adversarial contexts. These findings highlight the potential of targeted bias mitigation strategies to enhance the robustness and reliability of natural language understanding systems.