Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DTS-SQL: Decomposed Text-to-SQL with Small Large Language Models

Mohammadreza Pourreza, Davood Rafiei

TL;DR

The paper tackles privacy and cost concerns in Text-to-SQL by introducing a two-stage decomposed fine-tuning scheme that isolates schema linking from SQL generation, using two 7B open-source LLMs (DeepSeek 7B and Mistral 7B). This approach achieves 3–7 percentage-point improvements in execution accuracy over single-stage fine-tuning on Spider and Spider-Syn, attaining state-of-the-art development-set performance among open-source methods and competitive results with GPT-4 prompting on test sets. The study provides detailed analyses of schema-linking performance (around 90% exact-set-match) and discusses remaining gaps to upper-bound performance, offering a practical pathway for privacy-preserving, open-source Text-to-SQL systems. The work suggests future directions in enhancing schema linking, leveraging retrieval or hybrid prompting, and further narrowing the end-to-end gap between open-source and proprietary models.

Abstract

Leading models for the text-to-SQL task heavily rely on proprietary Large Language Models (LLMs), posing concerns over data privacy. Closing the performance gap between small open-source models and large proprietary models is crucial to mitigate this reliance. To this end, we introduce a novel two-stage fine-tuning approach that decomposes the task into two simpler tasks. Through comprehensive evaluation on two large cross-domain datasets and two small LLMs, we show that this approach improves execution accuracy by 3 to 7 percent, effectively aligning the performance of open-source models with their proprietary counterparts.

DTS-SQL: Decomposed Text-to-SQL with Small Large Language Models

TL;DR

The paper tackles privacy and cost concerns in Text-to-SQL by introducing a two-stage decomposed fine-tuning scheme that isolates schema linking from SQL generation, using two 7B open-source LLMs (DeepSeek 7B and Mistral 7B). This approach achieves 3–7 percentage-point improvements in execution accuracy over single-stage fine-tuning on Spider and Spider-Syn, attaining state-of-the-art development-set performance among open-source methods and competitive results with GPT-4 prompting on test sets. The study provides detailed analyses of schema-linking performance (around 90% exact-set-match) and discusses remaining gaps to upper-bound performance, offering a practical pathway for privacy-preserving, open-source Text-to-SQL systems. The work suggests future directions in enhancing schema linking, leveraging retrieval or hybrid prompting, and further narrowing the end-to-end gap between open-source and proprietary models.

Abstract

Leading models for the text-to-SQL task heavily rely on proprietary Large Language Models (LLMs), posing concerns over data privacy. Closing the performance gap between small open-source models and large proprietary models is crucial to mitigate this reliance. To this end, we introduce a novel two-stage fine-tuning approach that decomposes the task into two simpler tasks. Through comprehensive evaluation on two large cross-domain datasets and two small LLMs, we show that this approach improves execution accuracy by 3 to 7 percent, effectively aligning the performance of open-source models with their proprietary counterparts.
Paper Structure (20 sections, 3 equations, 3 figures, 6 tables)

This paper contains 20 sections, 3 equations, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Supervised fine-tuning for Text-to-SQL
  4. Decomposed Supervised Fine-tuning
  5. Schema-linking Fine-tuning
  6. SQL Generation Fine-tuning
  7. Experiments
  8. Models
  9. Hyperparameters
  10. Datasets
  11. Metrics
  12. Results
  13. Spider test set
  14. Spider dev set
  15. Schema-linking Performance
  16. ...and 5 more sections

Figures (3)

  • Figure 1: The prompt used for SQL generation. The database schema is where we put the tables representations.
  • Figure 2: The prompt used for Schema linking. The database schema is where we put the tables representations.
  • Figure 3: A sample table representation. All of the table in a database are represented as above and used in the prompts.