Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BASE-SQL: A powerful open source Text-To-SQL baseline approach

Lei Sheng, Shuai-Shuai Xu, Wei Xie

TL;DR

The paper tackles Text-to-SQL in privacy- and cost-constrained settings by proposing BASE-SQL, a pipeline-based baseline that relies on open-source LLMs. It introduces four components (Schema Linking, Candidate SQL Generate, SQL Revision, SQL Merge Revision) and leverages M-Schema representations with LoRA-finetuned models to achieve accurate SQL generation efficiently. On Spider and BIRD, BASE-SQL delivers competitive or superior results compared with open-source baselines and rivals some closed-source models, while requiring only about five LLM calls per SQL. This work provides a reproducible, low-cost baseline that balances accuracy, privacy, and practical deployment potential for real-world Text-to-SQL applications.

Abstract

The conversion of natural language into SQL language for querying databases (Text-to-SQL) has broad application prospects and has attracted widespread attention. At present, the mainstream Text-to-SQL methods are mainly divided into in-context learning (ICL) based methods and supervised fine-tuning (SFT) based methods. ICL-based methods can achieve relatively good results thanks to the use of the most advanced closed-source models. However, in real-world application scenarios, factors such as data privacy, SQL generation efficiency and cost need to be considered. SFT-based methods have certain advantages. At present, methods based on fine-tuning of open source models lack easy-to-implement and effective (cost-effective) baseline methods. We propose a pipeline-based method using open source model fine-tuning, referred to as BASE-SQL, which includes four components: Schema Linking, Candidate SQL Generate, SQL Revision and SQL Merge Revision. Experimental results show that BASE-SQL uses the open source model Qwen2.5-Coder-32B-Instruct, and achieves an accuracy of 67.47% on the BIRD development set and 88.9% on the Spider test set, which is significantly better than other methods using open source models, and even exceeds several methods using the GPT-4o closed-source model. At the same time, BASE-SQL is easy to implement and highly efficient (on average, only five calls to the large language model are required to generate SQL once). The code will be open sourced at https://github.com/CycloneBoy/base_sql.

BASE-SQL: A powerful open source Text-To-SQL baseline approach

TL;DR

The paper tackles Text-to-SQL in privacy- and cost-constrained settings by proposing BASE-SQL, a pipeline-based baseline that relies on open-source LLMs. It introduces four components (Schema Linking, Candidate SQL Generate, SQL Revision, SQL Merge Revision) and leverages M-Schema representations with LoRA-finetuned models to achieve accurate SQL generation efficiently. On Spider and BIRD, BASE-SQL delivers competitive or superior results compared with open-source baselines and rivals some closed-source models, while requiring only about five LLM calls per SQL. This work provides a reproducible, low-cost baseline that balances accuracy, privacy, and practical deployment potential for real-world Text-to-SQL applications.

Abstract

The conversion of natural language into SQL language for querying databases (Text-to-SQL) has broad application prospects and has attracted widespread attention. At present, the mainstream Text-to-SQL methods are mainly divided into in-context learning (ICL) based methods and supervised fine-tuning (SFT) based methods. ICL-based methods can achieve relatively good results thanks to the use of the most advanced closed-source models. However, in real-world application scenarios, factors such as data privacy, SQL generation efficiency and cost need to be considered. SFT-based methods have certain advantages. At present, methods based on fine-tuning of open source models lack easy-to-implement and effective (cost-effective) baseline methods. We propose a pipeline-based method using open source model fine-tuning, referred to as BASE-SQL, which includes four components: Schema Linking, Candidate SQL Generate, SQL Revision and SQL Merge Revision. Experimental results show that BASE-SQL uses the open source model Qwen2.5-Coder-32B-Instruct, and achieves an accuracy of 67.47% on the BIRD development set and 88.9% on the Spider test set, which is significantly better than other methods using open source models, and even exceeds several methods using the GPT-4o closed-source model. At the same time, BASE-SQL is easy to implement and highly efficient (on average, only five calls to the large language model are required to generate SQL once). The code will be open sourced at https://github.com/CycloneBoy/base_sql.

Paper Structure

This paper contains 30 sections, 3 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Schema Representation
  5. Schema Linking
  6. Candidate Generation
  7. SQL Revision
  8. SQL Merge Revision
  9. Experiments
  10. Experiments Setting
  11. Main Results
  12. Ablation Studies
  13. Analysis
  14. Impacts of Schema Linking
  15. Impacts of Different SQL Revision Methods
  16. ...and 15 more sections

Figures (3)

  • Figure 1: Overview of the proposed BASE-SQL framework, which consists of four components: 1) Schema linking: Identify the most relevant tables through a fine-tuned model; 2) Candidate Generation: Generate candidate SQL1 through a fine-tuned model; 3) SQL Revision: Use the all table schema, candidate SQL1 and its execution results to perform SQL correction and generate candidate SQL2 and candidate SQL3; 4) SQL Merge Revision: Use candidate SQL1, candidate SQL2 and candidate SQL3 to perform combined correction to generate the final SQL.
  • Figure 2: Examples of database schema representation methods: Code Representation, Code With Sample Representation, M-Schema Representation, and M-Schema With Sample Representation.
  • Figure 3: After using Qwen2.5-Coder of different sizes for Supervised Fine-tuning, the execution accuracy on the BIRD development set after evaluating different steps. The batch size of the 32B and 14B models is 8, and the batch size of the 7B model is 16. In the legend, 32B, 14B, and 7B use the M-Schema representation, while 32B_sample, 14B_sample, and 7B_sample use the M-Schema with sample representation.