Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FusionMind -- Improving question and answering with external context fusion

Shreyas Verma, Manoj Parmar, Palash Choudhary, Sanchita Porwal

TL;DR

This paper studies question answering with external context by comparing a fine-tuned LM approach on OpenBookQA against a LM+KG strategy based on QAGNN, and by testing additional factual context from OpenBookQA. It finds that incorporating factual context substantially boosts QA accuracy, while integrating knowledge graphs yields more modest gains, suggesting that contextual knowledge facts are highly impactful for reasoning in QA tasks. The work demonstrates the feasibility of external-context augmentation and highlights practical constraints such as token limits and compute, offering directions for scalable, context-rich QA systems with potential downstream benefits for virtual assistants and educational tools. Overall, contextual fact fusion emerges as a key lever for improving context-aware QA, with KG-based joint reasoning providing supplementary benefits under certain configurations.

Abstract

Answering questions using pre-trained language models (LMs) and knowledge graphs (KGs) presents challenges in identifying relevant knowledge and performing joint reasoning.We compared LMs (fine-tuned for the task) with the previously published QAGNN method for the Question-answering (QA) objective and further measured the impact of additional factual context on the QAGNN performance. The QAGNN method employs LMs to encode QA context and estimate KG node importance, and effectively update the question choice entity representations using Graph Neural Networks (GNNs). We further experimented with enhancing the QA context encoding by incorporating relevant knowledge facts for the question stem. The models are trained on the OpenbookQA dataset, which contains ~6000 4-way multiple choice questions and is widely used as a benchmark for QA tasks. Through our experimentation, we found that incorporating knowledge facts context led to a significant improvement in performance. In contrast, the addition of knowledge graphs to language models resulted in only a modest increase. This suggests that the integration of contextual knowledge facts may be more impactful for enhancing question answering performance compared to solely adding knowledge graphs.

FusionMind -- Improving question and answering with external context fusion

TL;DR

This paper studies question answering with external context by comparing a fine-tuned LM approach on OpenBookQA against a LM+KG strategy based on QAGNN, and by testing additional factual context from OpenBookQA. It finds that incorporating factual context substantially boosts QA accuracy, while integrating knowledge graphs yields more modest gains, suggesting that contextual knowledge facts are highly impactful for reasoning in QA tasks. The work demonstrates the feasibility of external-context augmentation and highlights practical constraints such as token limits and compute, offering directions for scalable, context-rich QA systems with potential downstream benefits for virtual assistants and educational tools. Overall, contextual fact fusion emerges as a key lever for improving context-aware QA, with KG-based joint reasoning providing supplementary benefits under certain configurations.

Abstract

Answering questions using pre-trained language models (LMs) and knowledge graphs (KGs) presents challenges in identifying relevant knowledge and performing joint reasoning.We compared LMs (fine-tuned for the task) with the previously published QAGNN method for the Question-answering (QA) objective and further measured the impact of additional factual context on the QAGNN performance. The QAGNN method employs LMs to encode QA context and estimate KG node importance, and effectively update the question choice entity representations using Graph Neural Networks (GNNs). We further experimented with enhancing the QA context encoding by incorporating relevant knowledge facts for the question stem. The models are trained on the OpenbookQA dataset, which contains ~6000 4-way multiple choice questions and is widely used as a benchmark for QA tasks. Through our experimentation, we found that incorporating knowledge facts context led to a significant improvement in performance. In contrast, the addition of knowledge graphs to language models resulted in only a modest increase. This suggests that the integration of contextual knowledge facts may be more impactful for enhancing question answering performance compared to solely adding knowledge graphs.
Paper Structure (16 sections, 4 figures, 4 tables)

This paper contains 16 sections, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. LITERATURE SURVEY/BASELINES
  3. Dataset description
  4. Data preparation
  5. Raw Data Statistics: Explain the dataset
  6. EXPERIMENTAL SETTINGS
  7. METHODS
  8. Approach 1: LM only method
  9. Approach 2: QAGNN method
  10. Experiments and Results
  11. Approach 1 results
  12. Approach 2 results
  13. Comparative analysis between the methods
  14. Conclusion
  15. Shortcomings of our work
  16. ...and 1 more sections

Figures (4)

  • Figure 1: Approach 1: LM Architecture
  • Figure 2: Approach 2: QAGNN Architecture
  • Figure 3: Sample question 1 deep-dive
  • Figure 4: Sample question 2 deep-dive