Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Decomposition for Enhancing Attention: Improving LLM-based Text-to-SQL through Workflow Paradigm

Yuanzhen Xie, Xinzhou Jin, Tao Xie, MingXiong Lin, Liang Chen, Chenyun Yu, Lei Cheng, ChengXiang Zhuo, Bo Hu, Zang Li

TL;DR

The paper tackles the challenge of attention diffusion in LLM-based text-to-SQL tasks by introducing a workflow prompting paradigm (DEA-SQL) that decomposes the problem into five modules: Information Determination, Classification & Hint, SQL Generation, Self-Correction, and Active Learning. This decomposition reduces interference from extraneous schema information, categorizes problem types to tailor prompts, and expands the model's problem-solving scope through self-correction and learning from errors. Empirical results on Spider, Spider Realistic, and Bird datasets show consistent improvements over baselines and achieve state-of-the-art performance on Spider Test, validating the approach's effectiveness and robustness. The work highlights practical trade-offs between cost and accuracy and offers a scalable, modular framework for enhancing LLM-based text-to-SQL systems in real-world settings.

Abstract

In-context learning of large-language models (LLMs) has achieved remarkable success in the field of natural language processing, while extensive case studies reveal that the single-step chain-of-thought prompting approach faces challenges such as attention diffusion and inadequate performance in complex tasks like text-to-SQL. To improve the contextual learning capabilities of LLMs in text-to-SQL, a workflow paradigm method is proposed, aiming to enhance the attention and problem-solving scope of LLMs through decomposition. Specifically, the information determination module for eliminating redundant information and the brand-new prompt structure based on problem classification greatly enhance the model's attention. Additionally, the inclusion of self-correction and active learning modules greatly expands the problem-solving scope of LLMs, hence improving the upper limit of LLM-based approaches. Extensive experiments conducted on three datasets demonstrate that our approach outperforms other methods by a significant margin. About 2-3 percentage point improvements compared to the existing baseline on the Spider Dev, Spider-Realistic, and Bird Dev datasets and new SOTA results on the Spider Test dataset are achieved. Our code is available on GitHub: \url{https://github.com/FlyingFeather/DEA-SQL}.

Decomposition for Enhancing Attention: Improving LLM-based Text-to-SQL through Workflow Paradigm

TL;DR

The paper tackles the challenge of attention diffusion in LLM-based text-to-SQL tasks by introducing a workflow prompting paradigm (DEA-SQL) that decomposes the problem into five modules: Information Determination, Classification & Hint, SQL Generation, Self-Correction, and Active Learning. This decomposition reduces interference from extraneous schema information, categorizes problem types to tailor prompts, and expands the model's problem-solving scope through self-correction and learning from errors. Empirical results on Spider, Spider Realistic, and Bird datasets show consistent improvements over baselines and achieve state-of-the-art performance on Spider Test, validating the approach's effectiveness and robustness. The work highlights practical trade-offs between cost and accuracy and offers a scalable, modular framework for enhancing LLM-based text-to-SQL systems in real-world settings.

Abstract

In-context learning of large-language models (LLMs) has achieved remarkable success in the field of natural language processing, while extensive case studies reveal that the single-step chain-of-thought prompting approach faces challenges such as attention diffusion and inadequate performance in complex tasks like text-to-SQL. To improve the contextual learning capabilities of LLMs in text-to-SQL, a workflow paradigm method is proposed, aiming to enhance the attention and problem-solving scope of LLMs through decomposition. Specifically, the information determination module for eliminating redundant information and the brand-new prompt structure based on problem classification greatly enhance the model's attention. Additionally, the inclusion of self-correction and active learning modules greatly expands the problem-solving scope of LLMs, hence improving the upper limit of LLM-based approaches. Extensive experiments conducted on three datasets demonstrate that our approach outperforms other methods by a significant margin. About 2-3 percentage point improvements compared to the existing baseline on the Spider Dev, Spider-Realistic, and Bird Dev datasets and new SOTA results on the Spider Test dataset are achieved. Our code is available on GitHub: \url{https://github.com/FlyingFeather/DEA-SQL}.
Paper Structure (42 sections, 4 figures, 8 tables)

This paper contains 42 sections, 4 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Information Determination
  4. Classification & Hint
  5. SQL Generation
  6. Self-Correction
  7. Active Learning
  8. Experiment
  9. Experiment Setup
  10. Implementation Details
  11. Datasets
  12. Spider yu2018spider.
  13. Spider Realistic deng2021structure.
  14. Bird li2024can.
  15. Evaluation
  16. ...and 27 more sections

Figures (4)

  • Figure 1: The overall structure of the DEA-SQL model.
  • Figure 2: DEA-SQL model's prompt structure in SQL Generation module.
  • Figure 3: The statistics of SQL errors based on information determination, classification & hint and SQL generation steps
  • Figure 4: Parameters analysis results.