Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SynTQA: Synergistic Table-based Question Answering via Mixture of Text-to-SQL and E2E TQA

Siyue Zhang, Anh Tuan Luu, Chen Zhao

TL;DR

This work systematically compares Text-to-SQL and end-to-end Table QA (E2E TQA), revealing complementary strengths across arithmetic reasoning, long-table handling, ambiguity, and complex content. It then introduces SynTQA, a model-agnostic ensemble that selects between Text-to-SQL and E2E TQA answers via feature-based or LLM-based selectors, yielding notable performance gains over single approaches. The study also demonstrates efficiency gains from partial SQL annotations and robustness improvements under adversarial perturbations, underscoring the practical viability of ensemble-based Table QA. Overall, SynTQA provides a scalable pathway to leverage diverse QA signals without heavy reliance on a single paradigm or expensive annotations.

Abstract

Text-to-SQL parsing and end-to-end question answering (E2E TQA) are two main approaches for Table-based Question Answering task. Despite success on multiple benchmarks, they have yet to be compared and their synergy remains unexplored. In this paper, we identify different strengths and weaknesses through evaluating state-of-the-art models on benchmark datasets: Text-to-SQL demonstrates superiority in handling questions involving arithmetic operations and long tables; E2E TQA excels in addressing ambiguous questions, non-standard table schema, and complex table contents. To combine both strengths, we propose a Synergistic Table-based Question Answering approach that integrate different models via answer selection, which is agnostic to any model types. Further experiments validate that ensembling models by either feature-based or LLM-based answer selector significantly improves the performance over individual models.

SynTQA: Synergistic Table-based Question Answering via Mixture of Text-to-SQL and E2E TQA

TL;DR

This work systematically compares Text-to-SQL and end-to-end Table QA (E2E TQA), revealing complementary strengths across arithmetic reasoning, long-table handling, ambiguity, and complex content. It then introduces SynTQA, a model-agnostic ensemble that selects between Text-to-SQL and E2E TQA answers via feature-based or LLM-based selectors, yielding notable performance gains over single approaches. The study also demonstrates efficiency gains from partial SQL annotations and robustness improvements under adversarial perturbations, underscoring the practical viability of ensemble-based Table QA. Overall, SynTQA provides a scalable pathway to leverage diverse QA signals without heavy reliance on a single paradigm or expensive annotations.

Abstract

Text-to-SQL parsing and end-to-end question answering (E2E TQA) are two main approaches for Table-based Question Answering task. Despite success on multiple benchmarks, they have yet to be compared and their synergy remains unexplored. In this paper, we identify different strengths and weaknesses through evaluating state-of-the-art models on benchmark datasets: Text-to-SQL demonstrates superiority in handling questions involving arithmetic operations and long tables; E2E TQA excels in addressing ambiguous questions, non-standard table schema, and complex table contents. To combine both strengths, we propose a Synergistic Table-based Question Answering approach that integrate different models via answer selection, which is agnostic to any model types. Further experiments validate that ensembling models by either feature-based or LLM-based answer selector significantly improves the performance over individual models.
Paper Structure (42 sections, 11 figures, 5 tables)

This paper contains 42 sections, 11 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Table Question Answering Task
  3. Text-to-SQL
  4. E2E TQA
  5. Evaluating Text-to-SQL and E2E TQA
  6. Experimental Setup
  7. Dataset.
  8. Model and Metric.
  9. Results
  10. Text-to-SQL is skilled at arithmetic operations.
  11. Text-to-SQL is adept at long tables.
  12. E2E TQA is robust to ambiguous questions and non-standard table schema.
  13. E2E TQA is flexible to process complex table content.
  14. Some questions cannot map to a SQL query.
  15. Text-to-SQL requires post-process the executed answers.
  16. ...and 27 more sections

Figures (11)

  • Figure 1: A demonstration of SOTA Table QA models' strengths in solving different types of table-based questions, followed by an overview of SynTQA. In a synergistic way, SynTQA aggregates candidate answers from Text-to-SQL and E2E TQA models, and then select the final answer. The answers in green color are the correct answers.
  • Figure 2: Error case analysis. $^\dagger$Arithmetic operation errors include questions with both long and short tables. Tables are regarded as long if their linearized sequences have more tokens than the Table QA model input length. The percentage numbers on the left indicate the quantity of error cases, and remaining percentage points correspond to other errors, such incorrect labels.
  • Figure 3: The impact of table size (i.e., number of rows) on the accuracy of E2E TQA, Text-to-SQL, and SynTQA(RF) on the the test set of WTQ. The x-axis represents the row number ranges, and the y-axis shows the average accuracy for each method.
  • Figure 4: WikiSQL test set accuracy versus the percentage amount of SQL annotations provided by Squall. Even an inferior Text-to-SQL model trained with a more limited set of SQL annotations can substantially enhance the E2E TQA model.
  • Figure 5: Breakdown of E2E TQA error cases (top) and Text-to-SQL error cases (bottom).
  • ...and 6 more figures