Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OpenSearch-SQL: Enhancing Text-to-SQL with Dynamic Few-shot and Consistency Alignment

Xiangjin Xie, Guangwei Xu, Lingyan Zhao, Ruijie Guo

TL;DR

OpenSearch-SQL addresses core challenges in Text-to-SQL by introducing a four-stage framework (Preprocessing, Extraction, Generation, Refinement) augmented with a consistency Alignment module to stabilize multi-agent outputs and reduce hallucinations. It further introduces a SQL-Like intermediate language and a dynamic self-taught Query-CoT-SQL few-shot strategy to enrich guidance without requiring post-training. Empirical results on BIRD and Spider demonstrate state-of-the-art execution accuracy and efficiency, validating the effectiveness and portability of the approach. The work highlights the potential of alignment-guided, non-finetuned LLM pipelines for robust cross-domain SQL generation and points to future directions in prompt optimization and broader application to multi-agent tasks.

Abstract

Although multi-agent collaborative Large Language Models (LLMs) have achieved significant breakthroughs in the Text-to-SQL task, their performance is still constrained by various factors. These factors include the incompleteness of the framework, failure to follow instructions, and model hallucination problems. To address these problems, we propose OpenSearch-SQL, which divides the Text-to-SQL task into four main modules: Preprocessing, Extraction, Generation, and Refinement, along with an Alignment module based on a consistency alignment mechanism. This architecture aligns the inputs and outputs of agents through the Alignment module, reducing failures in instruction following and hallucination. Additionally, we designed an intermediate language called SQL-Like and optimized the structured CoT based on SQL-Like. Meanwhile, we developed a dynamic few-shot strategy in the form of self-taught Query-CoT-SQL. These methods have significantly improved the performance of LLMs in the Text-to-SQL task. In terms of model selection, we directly applied the base LLMs without any post-training, thereby simplifying the task chain and enhancing the framework's portability. Experimental results show that OpenSearch-SQL achieves an execution accuracy(EX) of 69.3% on the BIRD development set, 72.28% on the test set, and a reward-based validity efficiency score (R-VES) of 69.36%, with all three metrics ranking first at the time of submission. These results demonstrate the comprehensive advantages of the proposed method in both effectiveness and efficiency.

OpenSearch-SQL: Enhancing Text-to-SQL with Dynamic Few-shot and Consistency Alignment

TL;DR

OpenSearch-SQL addresses core challenges in Text-to-SQL by introducing a four-stage framework (Preprocessing, Extraction, Generation, Refinement) augmented with a consistency Alignment module to stabilize multi-agent outputs and reduce hallucinations. It further introduces a SQL-Like intermediate language and a dynamic self-taught Query-CoT-SQL few-shot strategy to enrich guidance without requiring post-training. Empirical results on BIRD and Spider demonstrate state-of-the-art execution accuracy and efficiency, validating the effectiveness and portability of the approach. The work highlights the potential of alignment-guided, non-finetuned LLM pipelines for robust cross-domain SQL generation and points to future directions in prompt optimization and broader application to multi-agent tasks.

Abstract

Although multi-agent collaborative Large Language Models (LLMs) have achieved significant breakthroughs in the Text-to-SQL task, their performance is still constrained by various factors. These factors include the incompleteness of the framework, failure to follow instructions, and model hallucination problems. To address these problems, we propose OpenSearch-SQL, which divides the Text-to-SQL task into four main modules: Preprocessing, Extraction, Generation, and Refinement, along with an Alignment module based on a consistency alignment mechanism. This architecture aligns the inputs and outputs of agents through the Alignment module, reducing failures in instruction following and hallucination. Additionally, we designed an intermediate language called SQL-Like and optimized the structured CoT based on SQL-Like. Meanwhile, we developed a dynamic few-shot strategy in the form of self-taught Query-CoT-SQL. These methods have significantly improved the performance of LLMs in the Text-to-SQL task. In terms of model selection, we directly applied the base LLMs without any post-training, thereby simplifying the task chain and enhancing the framework's portability. Experimental results show that OpenSearch-SQL achieves an execution accuracy(EX) of 69.3% on the BIRD development set, 72.28% on the test set, and a reward-based validity efficiency score (R-VES) of 69.36%, with all three metrics ranking first at the time of submission. These results demonstrate the comprehensive advantages of the proposed method in both effectiveness and efficiency.

Paper Structure

This paper contains 43 sections, 3 equations, 4 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Framework.
  3. Preliminary
  4. Large Language Models
  5. Text-to-SQL
  6. Hallucination
  7. Methodology
  8. Alignments
  9. Self-Taught Fewshot
  10. Preprocessing
  11. Extraction
  12. Entity Extraction
  13. Values Retrieval.
  14. Column Filtering.
  15. Info Alignment
  16. ...and 28 more sections

Figures (4)

  • Figure 1: The basic framework of OpenSearch-SQL optimizes the Text-to-SQL task through a multi-agent collaboration approach based on consistency alignment.
  • Figure 2: Architecture of the Refinement: Optimizing SQLs through Correction and Consistency & Vote to get the best SQL.
  • Figure 3: Comparison of the impact of using Consistency & Vote across different difficulty levels.
  • Figure 4: Comparison of the impact of different numbers of candidates on model performance