Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DB-Explore: Automated Database Exploration and Instruction Synthesis for Text-to-SQL

Haoyuan Ma, Yongliang Shen, Hengwei Liu, Wenqi Zhang, Haolei Xu, Qiuying Peng, Jun Wang, Weiming Lu

TL;DR

DB-Explore addresses a core limitation of Text-to-SQL systems: insufficient database understanding. By constructing a DB Graph and performing multi-knowledge exploration (semantic extraction, structural pattern mining) followed by progressive instruction synthesis and rigorous filtering, it creates high-quality, database-consistent fine-tuning data. The approach yields strong results on SPIDER and BIRD with open-source 7B LLMs, achieving state-of-the-art performance at low computational cost and demonstrating significant data efficiency. The work highlights the practical impact of database-centric training for robust, domain-specific SQL generation and offers a scalable pathway toward more efficient fine-tuning and deployment.

Abstract

Recent text-to-SQL systems powered by large language models (LLMs) have demonstrated remarkable performance in translating natural language queries into SQL. However, these systems often struggle with complex database structures and domain-specific queries, as they primarily focus on enhancing logical reasoning and SQL syntax while overlooking the critical need for comprehensive database understanding. To address this limitation, we propose DB-Explore, a novel framework that systematically aligns LLMs with database knowledge through automated exploration and instruction synthesis. DB-Explore constructs database graphs to capture complex relational schemas, leverages GPT-4 to systematically mine structural patterns and semantic knowledge, and synthesizes instructions to distill this knowledge for efficient fine-tuning of LLMs. Our framework enables comprehensive database understanding through diverse sampling strategies and automated instruction generation, bridging the gap between database structures and language models. Experiments conducted on the SPIDER and BIRD benchmarks validate the effectiveness of DB-Explore, achieving an execution accuracy of 67.0% on BIRD and 87.8% on SPIDER. Notably, our open-source implementation based on Qwen2.5-Coder-7B achieves state-of-the-art results at minimal computational cost, outperforming several GPT-4-driven Text-to-SQL systems.

DB-Explore: Automated Database Exploration and Instruction Synthesis for Text-to-SQL

TL;DR

DB-Explore addresses a core limitation of Text-to-SQL systems: insufficient database understanding. By constructing a DB Graph and performing multi-knowledge exploration (semantic extraction, structural pattern mining) followed by progressive instruction synthesis and rigorous filtering, it creates high-quality, database-consistent fine-tuning data. The approach yields strong results on SPIDER and BIRD with open-source 7B LLMs, achieving state-of-the-art performance at low computational cost and demonstrating significant data efficiency. The work highlights the practical impact of database-centric training for robust, domain-specific SQL generation and offers a scalable pathway toward more efficient fine-tuning and deployment.

Abstract

Recent text-to-SQL systems powered by large language models (LLMs) have demonstrated remarkable performance in translating natural language queries into SQL. However, these systems often struggle with complex database structures and domain-specific queries, as they primarily focus on enhancing logical reasoning and SQL syntax while overlooking the critical need for comprehensive database understanding. To address this limitation, we propose DB-Explore, a novel framework that systematically aligns LLMs with database knowledge through automated exploration and instruction synthesis. DB-Explore constructs database graphs to capture complex relational schemas, leverages GPT-4 to systematically mine structural patterns and semantic knowledge, and synthesizes instructions to distill this knowledge for efficient fine-tuning of LLMs. Our framework enables comprehensive database understanding through diverse sampling strategies and automated instruction generation, bridging the gap between database structures and language models. Experiments conducted on the SPIDER and BIRD benchmarks validate the effectiveness of DB-Explore, achieving an execution accuracy of 67.0% on BIRD and 87.8% on SPIDER. Notably, our open-source implementation based on Qwen2.5-Coder-7B achieves state-of-the-art results at minimal computational cost, outperforming several GPT-4-driven Text-to-SQL systems.

Paper Structure

This paper contains 24 sections, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Database Graph Construction and Sampling
  5. Database Multi-Knowledge Exploration
  6. Semantic Knowledge Extraction
  7. Structural Pattern Mining
  8. Progressive Instruction Synthesis
  9. Instruction Filtering
  10. Supervised Fine-Tuning and Inference
  11. Experiments
  12. Benchmarks
  13. Evaluation Metrics
  14. Implementation Details
  15. Baselines
  16. ...and 9 more sections

Figures (4)

  • Figure 1: Overall framework of DB-Explore. Our framework operates through three principal phases: (1) Database Graph Construction, (2) Database Multi-Knowledge Exploration: semantic knowledge extraction, structural pattern mining, progressive instruction synthesis, and filtering mechanisms, and (3) Supervised Fine-Tuning executing database-adaptive model training. The synthesis phase systematically produces database-consistent training corpora through multi-stratum knowledge alignment.
  • Figure 2: Data volume analysis of different data synthesis methods on BIRD, CODES only reports the storage size for pre-training data
  • Figure 3: DB-Explore performance trend under different database exploration ratios.
  • Figure 4: Performance comparison of DB-Explore at different instruction generation stages on BIRD development set.