Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL

Zijin Hong, Zheng Yuan, Qinggang Zhang, Hao Chen, Junnan Dong, Feiran Huang, Xiao Huang

TL;DR

<3-5 sentence high-level summary>

Abstract

Generating accurate SQL from users' natural language questions (text-to-SQL) remains a long-standing challenge due to the complexities involved in user question understanding, database schema comprehension, and SQL generation. Traditional text-to-SQL systems, which combine human engineering and deep neural networks, have made significant progress. Subsequently, pre-trained language models (PLMs) have been developed for text-to-SQL tasks, achieving promising results. However, as modern databases and user questions grow more complex, PLMs with a limited parameter size often produce incorrect SQL. This necessitates more sophisticated and tailored optimization methods, which restricts the application of PLM-based systems. Recently, large language models (LLMs) have shown significant capabilities in natural language understanding as model scale increases. Thus, integrating LLM-based solutions can bring unique opportunities, improvements, and solutions to text-to-SQL research. In this survey, we provide a comprehensive review of existing LLM-based text-to-SQL studies. Specifically, we offer a brief overview of the technical challenges and evolutionary process of text-to-SQL. Next, we introduce the datasets and metrics designed to evaluate text-to-SQL systems. Subsequently, we present a systematic analysis of recent advances in LLM-based text-to-SQL. Finally, we make a summarization and discuss the remaining challenges in this field and suggest expectations for future research directions. All the related resources of LLM-based, including research papers, benchmarks, and open-source projects, are collected for the community in our repository: https://github.com/DEEP-PolyU/Awesome-LLM-based-Text2SQL.

Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL

TL;DR

<3-5 sentence high-level summary>

Abstract

Generating accurate SQL from users' natural language questions (text-to-SQL) remains a long-standing challenge due to the complexities involved in user question understanding, database schema comprehension, and SQL generation. Traditional text-to-SQL systems, which combine human engineering and deep neural networks, have made significant progress. Subsequently, pre-trained language models (PLMs) have been developed for text-to-SQL tasks, achieving promising results. However, as modern databases and user questions grow more complex, PLMs with a limited parameter size often produce incorrect SQL. This necessitates more sophisticated and tailored optimization methods, which restricts the application of PLM-based systems. Recently, large language models (LLMs) have shown significant capabilities in natural language understanding as model scale increases. Thus, integrating LLM-based solutions can bring unique opportunities, improvements, and solutions to text-to-SQL research. In this survey, we provide a comprehensive review of existing LLM-based text-to-SQL studies. Specifically, we offer a brief overview of the technical challenges and evolutionary process of text-to-SQL. Next, we introduce the datasets and metrics designed to evaluate text-to-SQL systems. Subsequently, we present a systematic analysis of recent advances in LLM-based text-to-SQL. Finally, we make a summarization and discuss the remaining challenges in this field and suggest expectations for future research directions. All the related resources of LLM-based, including research papers, benchmarks, and open-source projects, are collected for the community in our repository: https://github.com/DEEP-PolyU/Awesome-LLM-based-Text2SQL.
Paper Structure (52 sections, 5 equations, 3 figures, 3 tables)

This paper contains 52 sections, 5 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Introduction
  3. Overview
  4. Challenges in Text-to-SQL
  5. Linguistic Complexity and Ambiguity
  6. Schema Understanding and Representation
  7. Rare and Complex SQL Operations
  8. Cross-Domain Generalization
  9. Evolutionary Process
  10. Rule-based Methods
  11. Deep Learning-based Approaches
  12. PLM-based Implementation
  13. LLM-based Implementation
  14. Benchmarks & evalution
  15. Datasets
  16. ...and 37 more sections

Figures (3)

  • Figure 1: An example of LLM-based text-to-SQL is selected from the BIRD li2023BIRD dataset. A user asks a question about football leagues. The LLM takes this question along with the schema of the corresponding database as input and generates an SQL query as output. The generated SQL query can be executed in the database, retrieves the content "The 5 leagues with the highest matches", providing the answer to the user's question.
  • Figure 2: A sketch of text-to-SQL research evolution outlines the advancement of implementation paradigms, each represented with different color backgrounds. Before 2023, the focus is on a selection of representative traditional studies. However, from 2023 onward, the emphasis shifts to the rapid advancements driven by LLMs, marking a significant acceleration in the field.
  • Figure 3: Taxonomy tree of research in LLM-based text-to-SQL, taking inspiration from xu2023large; nodes are ordered by release time.