Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TTQA-RS- A break-down prompting approach for Multi-hop Table-Text Question Answering with Reasoning and Summarization

Jayetri Bardhan, Bushi Xiao, Daisy Zhe Wang

TL;DR

A Retrieval Augmented Generation (RAG) based model - TTQA-RS is proposed - A break-down prompting approach for Multi-hop Table-Text Question Answering with Reasoning and Summarization and achieves state-of-the-art performance for prompting-based methods on multi-hop table-text QA.

Abstract

Question answering (QA) over tables and text has gained much popularity over the years. Multi-hop table-text QA requires multiple hops between the table and text, making it a challenging QA task. Although several works have attempted to solve the table-text QA task, most involve training the models and requiring labeled data. In this paper, we have proposed a Retrieval Augmented Generation (RAG) based model - TTQA-RS: A break-down prompting approach for Multi-hop Table-Text Question Answering with Reasoning and Summarization. Our model uses an enhanced retriever for table-text information retrieval and uses augmented knowledge, including table-text summary with decomposed sub-questions with answers for a reasoning-based table-text QA. Using open-source language models, our model outperformed all existing prompting methods for table-text QA tasks on existing table-text QA datasets, such as HybridQA and OTT-QA's development set. Our experiments demonstrate the potential of prompt-based approaches using open-source LLMs. Additionally, by using LLaMA3-70B, our model achieved state-of-the-art performance for prompting-based methods on multi-hop table-text QA.

TTQA-RS- A break-down prompting approach for Multi-hop Table-Text Question Answering with Reasoning and Summarization

TL;DR

A Retrieval Augmented Generation (RAG) based model - TTQA-RS is proposed - A break-down prompting approach for Multi-hop Table-Text Question Answering with Reasoning and Summarization and achieves state-of-the-art performance for prompting-based methods on multi-hop table-text QA.

Abstract

Question answering (QA) over tables and text has gained much popularity over the years. Multi-hop table-text QA requires multiple hops between the table and text, making it a challenging QA task. Although several works have attempted to solve the table-text QA task, most involve training the models and requiring labeled data. In this paper, we have proposed a Retrieval Augmented Generation (RAG) based model - TTQA-RS: A break-down prompting approach for Multi-hop Table-Text Question Answering with Reasoning and Summarization. Our model uses an enhanced retriever for table-text information retrieval and uses augmented knowledge, including table-text summary with decomposed sub-questions with answers for a reasoning-based table-text QA. Using open-source language models, our model outperformed all existing prompting methods for table-text QA tasks on existing table-text QA datasets, such as HybridQA and OTT-QA's development set. Our experiments demonstrate the potential of prompt-based approaches using open-source LLMs. Additionally, by using LLaMA3-70B, our model achieved state-of-the-art performance for prompting-based methods on multi-hop table-text QA.
Paper Structure (26 sections, 1 equation, 7 figures, 5 tables)

This paper contains 26 sections, 1 equation, 7 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Our Model
  4. System Overview
  5. Retriever
  6. Reader
  7. Table-text Summarization
  8. Question decomposition
  9. Entity type prediction of the expected answer
  10. Table-text QA of independent sub-questions
  11. Table-text QA of the original questions
  12. Experimental Setup
  13. Datasets
  14. Implementation details
  15. Baseline Models
  16. ...and 11 more sections

Figures (7)

  • Figure 1: Comparison between Standard prompting, Chain of Thought prompting, and the TTQA-RS model.
  • Figure 2: An overview of TTQA-RS framework. The brown dashed lines represent the retriever and the black dotted lines represent the reader for the table-text QA model.
  • Figure 3: Example of rows to sentences for a table
  • Figure 4: Example of our approach using TTQA-RS model
  • Figure 5: EM-score of HybridQA test set on different LLaMA models
  • ...and 2 more figures