ReEx-SQL: Reasoning with Execution-Aware Reinforcement Learning for Text-to-SQL

TL;DR

ReEx-SQL addresses the critical gap of not leveraging execution feedback during generation in Text-to-SQL. By interleaving execution with reasoning through stepwise rollouts, a composite reward, and tree-structured decoding, it enables dynamic revision of intermediate queries and hypotheses. The approach, trained with GRPO and execution-aware prompts, yields strong in-domain and cross-domain results at 7B scale, with substantial efficiency gains from tree-structured decoding. This work demonstrates that real-time execution signals can meaningfully enhance semantic fidelity and robustness of SQL generation in practical, resource-constrained settings.

Abstract

In Text-to-SQL, execution feedback is essential for guiding large language models (LLMs) to reason accurately and generate reliable SQL queries. However, existing methods treat execution feedback solely as a post-hoc signal for correction or selection, failing to integrate it into the generation process. This limitation hinders their ability to address reasoning errors as they occur, ultimately reducing query accuracy and robustness. To address this issue, we propose ReEx-SQL (Reasoning with Execution-Aware Reinforcement Learning), a framework for Text-to-SQL that enables models to interact with the database during decoding and dynamically adjust their reasoning based on execution feedback. ReEx-SQL introduces an execution-aware reasoning paradigm that interleaves intermediate SQL execution into reasoning paths, facilitating context-sensitive revisions. It achieves this through structured prompts with markup tags and a stepwise rollout strategy that integrates execution feedback into each stage of generation. To supervise policy learning, we develop a composite reward function that includes an exploration reward, explicitly encouraging effective database interaction. Additionally, ReEx-SQL adopts a tree-based decoding strategy to support exploratory reasoning, enabling dynamic expansion of alternative reasoning paths. Notably, ReEx-SQL achieves 88.8% on Spider and 64.9% on BIRD at the 7B scale, surpassing the standard reasoning baseline by 2.7% and 2.6%, respectively. It also shows robustness, achieving 85.2% on Spider-Realistic with leading performance. In addition, its tree-structured decoding improves efficiency and performance over linear decoding, reducing inference time by 51.9% on the BIRD development set.

Figures (14)

• Figure 1: Standard reasoning vs. Execution-aware reasoning. Standard reasoning (e.g., Chain-of-Thought DBLP:conf/nips/cot) generates a reasoning path without validation, making it prone to error accumulation, syntax errors, and semantic misalignment. In contrast, execution-aware reasoning validates each step via execution feedback, enabling the detection and correction of both syntactic and semantic errors.
• Figure 2: Comparison between linear-structured and tree-structured self-consistency decoding.
• Figure 3: Multi-dimensional comparison between ReEx-SQL and the baseline: (a) Training reward over steps; (b) Response length throughout training; (c) BIRD performance across difficulty levels; (d) Error type distribution on the BIRD Dev; (e) Spider performance across difficulty levels.
• Figure 4: Execution-aware reasoning prompt for ReEx-SQL.
• Figure 5: Standard reasoning prompt for baseline.