Bridging the Gap: Transforming Natural Language Questions into SQL Queries via Abstract Query Pattern and Contextual Schema Markup

TL;DR

This paper tackles two fundamental gaps in Text-to-SQL: the structural mapping gap and the lexical mapping gap. It introduces PAS-SQL, a prompt-based SQL generation pipeline that uses Abstract Query Pattern (AQP) to capture question structure and Contextual Schema Markup (CSM) to anchor relevant schema content, complemented by curated demonstrations and a SQL correction loop. A Chain-of-Thought variant further improves efficiency by reducing token consumption while preserving performance. Empirical results on Spider and BIRD show state-of-the-art execution accuracy, with 87.9% on Spider dev and 64.67% on BIRD dev, validating both the approach and its scalability to real-world databases. The work highlights the importance of targeted prompt design and retrieval strategies for complex Text-to-SQL tasks and suggests directions for efficient, scalable NL-to-SQL systems.

Abstract

Large language models have demonstrated excellent performance in many tasks, including Text-to-SQL, due to their powerful in-context learning capabilities. They are becoming the mainstream approach for Text-to-SQL. However, these methods still have a significant gap compared to human performance, especially on complex questions. As the complexity of questions increases, the gap between questions and SQLs increases. We identify two important gaps: the structural mapping gap and the lexical mapping gap. To tackle these two gaps, we propose PAS-SQL, an efficient SQL generation pipeline based on LLMs, which alleviates gaps through Abstract Query Pattern (AQP) and Contextual Schema Markup (CSM). AQP aims to obtain the structural pattern of the question by removing database-related information, which enables us to find structurally similar demonstrations. CSM aims to associate database-related text span in the question with specific tables or columns in the database, which alleviates the lexical mapping gap. Experimental results on the Spider and BIRD datasets demonstrate the effectiveness of our proposed method. Specifically, PAS-SQL + GPT-4o sets a new state-of-the-art on the Spider benchmark with an execution accuracy of 87.9\%, and achieves leading results on the BIRD dataset with an execution accuracy of 64.67\%.

Figures (4)

• Figure 1: PAS-SQL alleviates gaps through AQP and CSM. AQP and CSM denote Abstract Query Pattern and Contextual Schema Markup respectively.
• Figure 2: Overall framework of proposed PAS-SQL.
• Figure 3: GLM-4' results on various few-shot numbers.
• Figure 4: Results of few-shot example retrieval.