Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LiCQA : A Lightweight Complex Question Answering System

Sourav Saha, Dwaipayan Roy, Mandar Mitra

TL;DR

LiCQA is presented, an unsupervised question answer- ing model that works primarily on the basis of corpus evidence that significantly outperforms these two state-of-the-art QA systems on benchmark data with noteworthy reduction in latency.

Abstract

Over the last twenty years, significant progress has been made in designing and implementing Question Answering (QA) systems. However, addressing complex questions, the answers to which are spread across multiple documents, remains a challenging problem. Recent QA systems that are designed to handle complex questions work either on the basis of knowledge graphs, or utilise contem- porary neural models that are expensive to train, in terms of both computational resources and the volume of training data required. In this paper, we present LiCQA, an unsupervised question answer- ing model that works primarily on the basis of corpus evidence. We empirically compare the effectiveness and efficiency of LiCQA with two recently presented QA systems, which are based on different underlying principles. The results of our experiments show that LiCQA significantly outperforms these two state-of-the-art systems on benchmark data with noteworthy reduction in latency.

LiCQA : A Lightweight Complex Question Answering System

TL;DR

LiCQA is presented, an unsupervised question answer- ing model that works primarily on the basis of corpus evidence that significantly outperforms these two state-of-the-art QA systems on benchmark data with noteworthy reduction in latency.

Abstract

Over the last twenty years, significant progress has been made in designing and implementing Question Answering (QA) systems. However, addressing complex questions, the answers to which are spread across multiple documents, remains a challenging problem. Recent QA systems that are designed to handle complex questions work either on the basis of knowledge graphs, or utilise contem- porary neural models that are expensive to train, in terms of both computational resources and the volume of training data required. In this paper, we present LiCQA, an unsupervised question answer- ing model that works primarily on the basis of corpus evidence. We empirically compare the effectiveness and efficiency of LiCQA with two recently presented QA systems, which are based on different underlying principles. The results of our experiments show that LiCQA significantly outperforms these two state-of-the-art systems on benchmark data with noteworthy reduction in latency.
Paper Structure (20 sections, 1 equation, 4 figures, 9 tables)

This paper contains 20 sections, 1 equation, 4 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. $\mathsf{LiCQA}$: A lightweight complex question answering model
  4. Question Type Classification
  5. Answer Extraction and Filtering
  6. Answer Scoring
  7. Answer Ranking
  8. Experimental Setup
  9. Baselines
  10. Datasets
  11. Evaluation metrics
  12. $\mathsf{LiCQA}$ configuration
  13. Result and Discussions
  14. Analysis of pipeline components
  15. End-to-end evaluation
  16. ...and 5 more sections

Figures (4)

  • Figure 1: Architecture of the $\mathsf{LiCQA}$ system.
  • Figure 2: Performance analysis of $\mathsf{LiCQA}$, DrQA and QUEST on CQ-W and CQ-T question set on the Top10 dataset. First and second barplots (shown in 'khaki' and 'olive' color) denote QUEST result with normal and tie-aware metrics respectively. Similarly fourth and fifth barplots (shown in 'skyblue' and 'steelblue' color) denote $\mathsf{LiCQA}$ result with normal and tie-aware metrics respectively. Middle bar with teal color presents result of DrQA (as it retrieves a single entity per rank position).
  • Figure 3: A per-query performance analysis between $\mathsf{LiCQA}$ and QUEST on CQ-W question set for Top10 dataset. Bars above x-axis correspond to questions for which $\mathsf{LiCQA}$ has achieved better performance than QUEST.
  • Figure 4: A per-query performance analysis between $\mathsf{LiCQA}$ and QUEST on CQ-T question set for Top10 dataset. Bars above x-axis correspond to questions for which $\mathsf{LiCQA}$ has achieved better performance than QUEST.