Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RetinaQA: A Robust Knowledge Base Question Answering Model for both Answerable and Unanswerable Questions

Prayushi Faldu, Indrajit Bhattacharya, Mausam

TL;DR

This work proposes RetinaQA, a new KBQA model that unifies two key ideas in a single KBQA architecture: discrimination over candidate logical forms, rather than generating these, for handling schema-related unanswerability, and sketch-filling-based construction of candidate logical forms for handling data-related unaswerability.

Abstract

An essential requirement for a real-world Knowledge Base Question Answering (KBQA) system is the ability to detect the answerability of questions when generating logical forms. However, state-of-the-art KBQA models assume all questions to be answerable. Recent research has found that such models, when superficially adapted to detect answerability, struggle to satisfactorily identify the different categories of unanswerable questions, and simultaneously preserve good performance for answerable questions. Towards addressing this issue, we propose RetinaQA, a new KBQA model that unifies two key ideas in a single KBQA architecture: (a) discrimination over candidate logical forms, rather than generating these, for handling schema-related unanswerability, and (b) sketch-filling-based construction of candidate logical forms for handling data-related unaswerability. Our results show that RetinaQA significantly outperforms adaptations of state-of-the-art KBQA models in handling both answerable and unanswerable questions and demonstrates robustness across all categories of unanswerability. Notably, RetinaQA also sets a new state-of-the-art for answerable KBQA, surpassing existing models.

RetinaQA: A Robust Knowledge Base Question Answering Model for both Answerable and Unanswerable Questions

TL;DR

This work proposes RetinaQA, a new KBQA model that unifies two key ideas in a single KBQA architecture: discrimination over candidate logical forms, rather than generating these, for handling schema-related unanswerability, and sketch-filling-based construction of candidate logical forms for handling data-related unaswerability.

Abstract

An essential requirement for a real-world Knowledge Base Question Answering (KBQA) system is the ability to detect the answerability of questions when generating logical forms. However, state-of-the-art KBQA models assume all questions to be answerable. Recent research has found that such models, when superficially adapted to detect answerability, struggle to satisfactorily identify the different categories of unanswerable questions, and simultaneously preserve good performance for answerable questions. Towards addressing this issue, we propose RetinaQA, a new KBQA model that unifies two key ideas in a single KBQA architecture: (a) discrimination over candidate logical forms, rather than generating these, for handling schema-related unanswerability, and (b) sketch-filling-based construction of candidate logical forms for handling data-related unaswerability. Our results show that RetinaQA significantly outperforms adaptations of state-of-the-art KBQA models in handling both answerable and unanswerable questions and demonstrates robustness across all categories of unanswerability. Notably, RetinaQA also sets a new state-of-the-art for answerable KBQA, surpassing existing models.
Paper Structure (23 sections, 3 equations, 1 figure, 13 tables)

This paper contains 23 sections, 3 equations, 1 figure, 13 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. The Problem and Our Solution
  4. KBQA with Unanswerability
  5. The RetinaQA Model
  6. Entity Linker:
  7. Logical Form Retriever:
  8. Logical Form Constructor:
  9. Logical Form Discriminator:
  10. Experiments
  11. Experimental Setup
  12. Results for KBQA with Unanswerability
  13. Results for Answerable-only KBQA
  14. Ablation Study
  15. Error Analysis
  16. ...and 8 more sections

Figures (1)

  • Figure 1: RetinaQA Architecture showing different components illustrated with an example question. Symbols $l$ and $s$ represent candidate logical form and its score as Discriminator output, $L$ the output logical form, $A$ the final answer, $l(1)$ the top ranked logical form, Ex($l$) the answer obtained by executing logical form $l$. NK and NA are special symbols indicating No Knowledge (for logical form) and No Answer. The logical form in red under LF Retriever would not be found if data element (C. Manning, works at, Stanford) is missing in the KB, and additionally that in red under LF Integrator would not be found if relation works at is missing in KB schema and therefore not retrieved by the Schema Retriever. The candidate logical form in red under Discriminator would not be found if both of these are missing.