Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Track-SQL: Enhancing Generative Language Models with Dual-Extractive Modules for Schema and Context Tracking in Multi-turn Text-to-SQL

Bingfeng Chen, Shaobin Shi, Yongqi Luo, Boyan Xu, Ruichu Cai, Zhifeng Hao

TL;DR

A framework named Track-SQL, which enhances generative language models with dual-extractive modules designed to track schema and contextual changes in multi-turn Text-to-SQL, and achieves state-of-the-art performance on the SparC and CoSQL datasets.

Abstract

Generative language models have shown significant potential in single-turn Text-to-SQL. However, their performance does not extend equivalently to multi-turn Text-to-SQL. This is primarily due to generative language models' inadequacy in handling the complexities of context information and dynamic schema linking in multi-turn interactions. In this paper, we propose a framework named Track-SQL, which enhances generative language models with dual-extractive modules designed to track schema and contextual changes in multi-turn Text-to-SQL. Specifically, Track-SQL incorporates a \emph{Semantic-enhanced Schema Extractor} and a \emph{Schema-aware Context Extractor}. Experimental results demonstrate that Track-SQL achieves state-of-the-art performance on the SparC and CoSQL datasets. Furthermore, detailed ablation studies reveal that Track-SQL significantly improves execution accuracy in multi-turn interactions by 7.1\% and 9.55\% on these datasets, respectively. Our implementation will be open-sourced at https://github.com/DMIRLAB-Group/Track-SQL.

Track-SQL: Enhancing Generative Language Models with Dual-Extractive Modules for Schema and Context Tracking in Multi-turn Text-to-SQL

TL;DR

A framework named Track-SQL, which enhances generative language models with dual-extractive modules designed to track schema and contextual changes in multi-turn Text-to-SQL, and achieves state-of-the-art performance on the SparC and CoSQL datasets.

Abstract

Generative language models have shown significant potential in single-turn Text-to-SQL. However, their performance does not extend equivalently to multi-turn Text-to-SQL. This is primarily due to generative language models' inadequacy in handling the complexities of context information and dynamic schema linking in multi-turn interactions. In this paper, we propose a framework named Track-SQL, which enhances generative language models with dual-extractive modules designed to track schema and contextual changes in multi-turn Text-to-SQL. Specifically, Track-SQL incorporates a \emph{Semantic-enhanced Schema Extractor} and a \emph{Schema-aware Context Extractor}. Experimental results demonstrate that Track-SQL achieves state-of-the-art performance on the SparC and CoSQL datasets. Furthermore, detailed ablation studies reveal that Track-SQL significantly improves execution accuracy in multi-turn interactions by 7.1\% and 9.55\% on these datasets, respectively. Our implementation will be open-sourced at https://github.com/DMIRLAB-Group/Track-SQL.
Paper Structure (27 sections, 16 equations, 4 figures, 16 tables)

This paper contains 27 sections, 16 equations, 4 figures, 16 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Model Overview
  4. Semantic-enhanced Schema Extractor
  5. Schema-aware Context Extractor
  6. SQL Generation Fine-tuning
  7. Experiments
  8. Experimental Setup
  9. Main Results
  10. Ablation Studies
  11. Execution Time Analysis
  12. Related Work
  13. Conclusion
  14. Appendix A
  15. Further Ablation Studies
  16. ...and 12 more sections

Figures (4)

  • Figure 1: The overall framework of Dual-Extractive Modules for Schema and Context Tracking. The framework trains a schema item classification model and an SQL generator. Based on the former, we construct a Semantic-enhanced Schema Extractor and a Schema-aware Context Extractor. The extraction results from these two extractors are utilized for subsequent training of the SQL generation model. The core idea of Track-SQL is to reduce the gap between the input and the target SQL before entering the multi-turn SQL generation phase by means of dynamic schema linking and context information extraction.
  • Figure 2: The results of the ablation study on the Codellama 7B+Track-SQL model on the SparC and CoSQL dev sets (calculated using the multi-turn TS metrics).
  • Figure 3: The results of the ablation study on the DeepSeek 7B+Track-SQL model on the SparC and CoSQL dev sets (calculated using the multi-turn TS metrics).
  • Figure 4: The results of the ablation study on the Mistral 7B+Track-SQL model on the SparC and CoSQL dev sets (calculated using the multi-turn TS metrics).