Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SQL-Trail: Multi-Turn Reinforcement Learning with Interleaved Feedback for Text-to-SQL

Harper Hua, Zhen Han, Zhengyuan Shen, Jeremy Lee, Patrick Guan, Qi Zhu, Sullam Jeoung, Yueyan Chen, Yunfei Bai, Shuai Wang, Vassilis Ioannidis, Huzefa Rangwala

TL;DR

SQL-Trail reframes Text-to-SQL as an interactive, multi-turn agent that learns through execution feedback rather than a single pass. By combining adaptive turn budgeting with a dense, six-term reward design and a GRPO-based RL objective, it enables smaller open-source models to achieve state-of-the-art data efficiency and robust cross-domain generalization on Spider and BIRD. The two-stage training pipeline (SFT then RL) paired with careful data curation demonstrates that iterative reasoning with a SQL engine yields substantial gains in accuracy and reliability over traditional single-pass RL and SFT approaches. This interactive, tool-augmented workflow has practical implications for scalable, data-efficient Text-to-SQL systems in real-world databases, albeit with increased inference cost and the need for an accessible execution environment.

Abstract

While large language models (LLMs) have substantially improved Text-to-SQL generation, a pronounced gap remains between AI systems and human experts on challenging benchmarks such as BIRD-SQL. We argue this gap stems largely from the prevailing single-pass paradigm, which lacks the iterative reasoning, schema exploration, and error-correction behaviors that humans naturally employ. To address this limitation, we introduce SQL-Trail, a multi-turn reinforcement learning (RL) agentic framework for Text-to-SQL. Rather than producing a query in one shot, SQL-Trail interacts with the database environment and uses execution feedback to iteratively refine its predictions. Our approach centers on two key ideas: (i) an adaptive turn-budget allocation mechanism that scales the agent's interaction depth to match question difficulty, and (ii) a composite reward panel that jointly incentivizes SQL correctness and efficient exploration. Across benchmarks, SQL-Trail sets a new state of the art and delivers strong data efficiency--up to 18x higher than prior single-pass RL state-of-the-art methods. Notably, our 7B and 14B models outperform substantially larger proprietary systems by 5% on average, underscoring the effectiveness of interactive, agentic workflows for robust Text-to-SQL generation.

SQL-Trail: Multi-Turn Reinforcement Learning with Interleaved Feedback for Text-to-SQL

TL;DR

SQL-Trail reframes Text-to-SQL as an interactive, multi-turn agent that learns through execution feedback rather than a single pass. By combining adaptive turn budgeting with a dense, six-term reward design and a GRPO-based RL objective, it enables smaller open-source models to achieve state-of-the-art data efficiency and robust cross-domain generalization on Spider and BIRD. The two-stage training pipeline (SFT then RL) paired with careful data curation demonstrates that iterative reasoning with a SQL engine yields substantial gains in accuracy and reliability over traditional single-pass RL and SFT approaches. This interactive, tool-augmented workflow has practical implications for scalable, data-efficient Text-to-SQL systems in real-world databases, albeit with increased inference cost and the need for an accessible execution environment.

Abstract

While large language models (LLMs) have substantially improved Text-to-SQL generation, a pronounced gap remains between AI systems and human experts on challenging benchmarks such as BIRD-SQL. We argue this gap stems largely from the prevailing single-pass paradigm, which lacks the iterative reasoning, schema exploration, and error-correction behaviors that humans naturally employ. To address this limitation, we introduce SQL-Trail, a multi-turn reinforcement learning (RL) agentic framework for Text-to-SQL. Rather than producing a query in one shot, SQL-Trail interacts with the database environment and uses execution feedback to iteratively refine its predictions. Our approach centers on two key ideas: (i) an adaptive turn-budget allocation mechanism that scales the agent's interaction depth to match question difficulty, and (ii) a composite reward panel that jointly incentivizes SQL correctness and efficient exploration. Across benchmarks, SQL-Trail sets a new state of the art and delivers strong data efficiency--up to 18x higher than prior single-pass RL state-of-the-art methods. Notably, our 7B and 14B models outperform substantially larger proprietary systems by 5% on average, underscoring the effectiveness of interactive, agentic workflows for robust Text-to-SQL generation.
Paper Structure (56 sections, 13 equations, 10 figures, 7 tables, 1 algorithm)

This paper contains 56 sections, 13 equations, 10 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. SQL-Trail
  4. Overview
  5. Generation with Multi-Turn SQL Engine Calling
  6. Training phase 1: Supervised Fine-Tuning
  7. Training Phase 2: Multi-turn Reinforcement Learning with a SQL Engine
  8. Reinforcement Learning Formulation
  9. Multi-turn Reward Design
  10. Final execution reward.
  11. Turn number reward.
  12. Auxiliary shaping rewards.
  13. Total reward.
  14. Experiment Setup
  15. Model setup
  16. ...and 41 more sections

Figures (10)

  • Figure 1: Illustration of core Text-to-SQL challenges and the shift from single-pass to multi-turn generation.
  • Figure 2: Overview of SQL-Trail. The top panel demonstrates the multi-turn Text-to-SQL interaction process, and the bottom panel outlines the unified RL training pipeline.
  • Figure 3: Training dynamics for Single-Pass vs. Multi-Turn RL. Curves represent the 10-step moving average of raw rewards (top) and Pass@6 accuracy (bottom).
  • Figure 4: Performance comparison on the BIRD-dev set across three difficulty levels. The top panel reports the average number of turns, while the bottom panel displays the Average Execution Accuracy (EX). The models evaluated are SQL-Trail RL, SQL-Trail SFT, SQL-Trail RL w/o turn reward, Qwen2.5-Coder Agent, and Sonnet Agent.
  • Figure 5: Reasoning efficiency analysis on BIRD-dev. We plot Execution Accuracy against the average quantity of reasoning (measured in characters within <reasoning> tags) across different difficulty levels
  • ...and 5 more figures