IncSQL: Training Incremental Text-to-SQL Parsers with Non-Deterministic Oracles

TL;DR

The paper tackles NL2SQL by addressing the issue that multiple SQL queries can express the same intent. It introduces an incremental sequence-to-action parser trained with non-deterministic oracles to account for alternative correct continuations, and employs greedy decoding with execution-guided enhancements. On WikiSQL, it achieves a 2.1% absolute gain over static-oracle baselines and reaches 87.1% test execution accuracy when combined with execution-guided decoding, setting a new state-of-the-art. The approach is also validated on ATIS, supporting its applicability beyond WikiSQL.

Abstract

We present a sequence-to-action parsing approach for the natural language to SQL task that incrementally fills the slots of a SQL query with feasible actions from a pre-defined inventory. To account for the fact that typically there are multiple correct SQL queries with the same or very similar semantics, we draw inspiration from syntactic parsing techniques and propose to train our sequence-to-action models with non-deterministic oracles. We evaluate our models on the WikiSQL dataset and achieve an execution accuracy of 83.7% on the test set, a 2.1% absolute improvement over the models trained with traditional static oracles assuming a single correct target SQL query. When further combined with the execution-guided decoding strategy, our model sets a new state-of-the-art performance at an execution accuracy of 87.1%.

Figures (1)

• Figure 1: Our running example. The input is a natural language question and a table schema, and the output is an executable SQL query. Table contents are shown here, but unknown to our models.