Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Weakly Supervised Text-to-SQL Parsing through Question Decomposition

Tomer Wolfson, Daniel Deutch, Jonathan Berant

TL;DR

This work introduces a weakly supervised approach to train Text-to-SQL parsers without SQL annotations by leveraging QDMR as an intermediate representation and question-answer supervision. It presents a complete SQL synthesis pipeline consisting of phrase-DB linking, join-path inference, and a QDMR-to-SQL mapper, followed by execution-guided candidate selection to ensure correctness. Across five benchmarks, the method achieves 77.8% SQL synthesis coverage and enables training of competitive NL-to-SQL models, including strong cross-database generalization, with only annotated or predicted QDMRs and answers. The results demonstrate that near-gold performance is attainable with zero SQL annotations, underscoring the practicality and scalability of crowd-sourced supervision for semantic parsing in databases.

Abstract

Text-to-SQL parsers are crucial in enabling non-experts to effortlessly query relational data. Training such parsers, by contrast, generally requires expertise in annotating natural language (NL) utterances with corresponding SQL queries. In this work, we propose a weak supervision approach for training text-to-SQL parsers. We take advantage of the recently proposed question meaning representation called QDMR, an intermediate between NL and formal query languages. Given questions, their QDMR structures (annotated by non-experts or automatically predicted), and the answers, we are able to automatically synthesize SQL queries that are used to train text-to-SQL models. We test our approach by experimenting on five benchmark datasets. Our results show that the weakly supervised models perform competitively with those trained on annotated NL-SQL data. Overall, we effectively train text-to-SQL parsers, while using zero SQL annotations.

Weakly Supervised Text-to-SQL Parsing through Question Decomposition

TL;DR

This work introduces a weakly supervised approach to train Text-to-SQL parsers without SQL annotations by leveraging QDMR as an intermediate representation and question-answer supervision. It presents a complete SQL synthesis pipeline consisting of phrase-DB linking, join-path inference, and a QDMR-to-SQL mapper, followed by execution-guided candidate selection to ensure correctness. Across five benchmarks, the method achieves 77.8% SQL synthesis coverage and enables training of competitive NL-to-SQL models, including strong cross-database generalization, with only annotated or predicted QDMRs and answers. The results demonstrate that near-gold performance is attainable with zero SQL annotations, underscoring the practicality and scalability of crowd-sourced supervision for semantic parsing in databases.

Abstract

Text-to-SQL parsers are crucial in enabling non-experts to effortlessly query relational data. Training such parsers, by contrast, generally requires expertise in annotating natural language (NL) utterances with corresponding SQL queries. In this work, we propose a weak supervision approach for training text-to-SQL parsers. We take advantage of the recently proposed question meaning representation called QDMR, an intermediate between NL and formal query languages. Given questions, their QDMR structures (annotated by non-experts or automatically predicted), and the answers, we are able to automatically synthesize SQL queries that are used to train text-to-SQL models. We test our approach by experimenting on five benchmark datasets. Our results show that the weakly supervised models perform competitively with those trained on annotated NL-SQL data. Overall, we effectively train text-to-SQL parsers, while using zero SQL annotations.
Paper Structure (38 sections, 3 figures, 14 tables, 1 algorithm)

This paper contains 38 sections, 3 figures, 14 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Weakly Supervised ML
  4. Question Decomposition
  5. Weakly Supervised SQL Synthesis
  6. Synthesizing SQL from QDMR
  7. Phrase DB Linking
  8. Join Path Inference
  9. QDMR to SQL Mapper
  10. Execution-guided SQL Candidate Search
  11. Experiments
  12. Setting
  13. Datasets
  14. Training
  15. SQL Synthesis Coverage
  16. ...and 23 more sections

Figures (3)

  • Figure 1: Our pipeline for training a Text-to-SQL model on data synthesized using weak supervision.
  • Figure 2: Previously mapped QDMR steps (with operations and arguments) used as nested SQL queries.
  • Figure 3: Handling Self-joins in QDMR mapping. The two FILTER steps have conflicting assignments to the same column and are identified as a self-join. This is resolved by using a nested query in the SQL of step 3.