Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning

TL;DR

This work proposes a model consisting of less than $500$M parameters, to act as an extremely efficient approximator, enhancing it with the ability to process schemata in a parallelised manner, and applies it to monolingual and cross-lingual benchmarks for semantic parsing, showing improvements over state-of-the-art baselines.

Abstract

We focus on Text-to-SQL semantic parsing from the perspective of retrieval-augmented generation. Motivated by challenges related to the size of commercial database schemata and the deployability of business intelligence solutions, we propose $\text{ASTReS}$ that dynamically retrieves input database information and uses abstract syntax trees to select few-shot examples for in-context learning. Furthermore, we investigate the extent to which an in-parallel semantic parser can be leveraged for generating approximated versions of the expected SQL queries, to support our retrieval. We take this approach to the extreme--we adapt a model consisting of less than $500$M parameters, to act as an extremely efficient approximator, enhancing it with the ability to process schemata in a parallelised manner. We apply $\text{ASTReS}$ to monolingual and cross-lingual benchmarks for semantic parsing, showing improvements over state-of-the-art baselines. Comprehensive experiments highlight the contribution of modules involved in this retrieval-augmented generation setting, revealing interesting directions for future work.

Figures (2)

• Figure 1: Execution accuracy scores on on the development set of Spider across different maximum numbers of columns per schema split, $r$. The results of our approach, using gpt-3.5-turbo, are presented across different Spider-query difficulty levels.
• Figure 2: Example of how the similarity between two different SQL queries is computed using normalised ASTs.