Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Training Table Question Answering via SQL Query Decomposition

Raphaël Mouravieff, Benjamin Piwowarski, Sylvain Lamprier

TL;DR

The paper tackles Table Question Answering by bridging semantic parsing and direct NL-to-answer methods through a tabular algebra. It translates NL questions and tables into a computational graph derived from an SQL-like parse, then uses partial execution to determine which portions are handled by the model versus external tools, enabling flexible intermediate supervision. Training follows a two-stage pre-training: first learn to translate SQL into a formal logical form, then fine-tune on NL questions, yielding robust generalization and strong denotation accuracy on WikiTableQuestions (WTQ). Empirical results show competitive or state-of-the-art performance with less aggressive data augmentation, and the study emphasizes robustness to perturbations and the value of ensembling diverse granularities of the algebra. This approach provides practical insights into what operations should be predicted by a generative model versus executed by an external algorithm, with potential applicability to LLM-facilitated table reasoning.

Abstract

Table Question-Answering involves both understanding the natural language query and grounding it in the context of the input table to extract the relevant information. In this context, many methods have highlighted the benefits of intermediate pre-training from SQL queries. However, while most approaches aim at generating final answers from inputs directly, we claim that there is better to do with SQL queries during training. By learning to imitate a restricted portion of SQL-like algebraic operations, we show that their execution flow provides intermediate supervision steps that allow increased generalization and structural reasoning compared with classical approaches of the field. Our study bridges the gap between semantic parsing and direct answering methods and provides useful insights regarding what types of operations should be predicted by a generative architecture or be preferably executed by an external algorithm.

Training Table Question Answering via SQL Query Decomposition

TL;DR

The paper tackles Table Question Answering by bridging semantic parsing and direct NL-to-answer methods through a tabular algebra. It translates NL questions and tables into a computational graph derived from an SQL-like parse, then uses partial execution to determine which portions are handled by the model versus external tools, enabling flexible intermediate supervision. Training follows a two-stage pre-training: first learn to translate SQL into a formal logical form, then fine-tune on NL questions, yielding robust generalization and strong denotation accuracy on WikiTableQuestions (WTQ). Empirical results show competitive or state-of-the-art performance with less aggressive data augmentation, and the study emphasizes robustness to perturbations and the value of ensembling diverse granularities of the algebra. This approach provides practical insights into what operations should be predicted by a generative model versus executed by an external algorithm, with potential applicability to LLM-facilitated table reasoning.

Abstract

Table Question-Answering involves both understanding the natural language query and grounding it in the context of the input table to extract the relevant information. In this context, many methods have highlighted the benefits of intermediate pre-training from SQL queries. However, while most approaches aim at generating final answers from inputs directly, we claim that there is better to do with SQL queries during training. By learning to imitate a restricted portion of SQL-like algebraic operations, we show that their execution flow provides intermediate supervision steps that allow increased generalization and structural reasoning compared with classical approaches of the field. Our study bridges the gap between semantic parsing and direct answering methods and provides useful insights regarding what types of operations should be predicted by a generative architecture or be preferably executed by an external algorithm.
Paper Structure (30 sections, 1 equation, 3 figures, 8 tables)

This paper contains 30 sections, 1 equation, 3 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Table Question Answering Architectures
  4. Output Strategies in Table QA
  5. Semantic Parsing
  6. Direct Answer Generation
  7. Hybrid Methods
  8. Model
  9. Tabular algebra
  10. Structures
  11. Operators
  12. Partial Execution of the computational graph
  13. Linearizing the Graph
  14. Pre-order vs post-order
  15. Using aliases
  16. ...and 15 more sections

Figures (3)

  • Figure 1: Overview of the different approaches for Table QA and their limits (a-c), along with our proposition (d)
  • Figure 2: Evolution of FDA (test set) for different model variants.
  • Figure 3: Evolution of FDA (test set) depending on the number of model variations in the ensemble. We either add models to the ensemble starting from the left (green) or the right (blue).