MTSQL-R1: Towards Long-Horizon Multi-Turn Text-to-SQL via Agentic Training

TL;DR

MTSQL-R1 advances conversational semantic parsing by introducing an agentic training framework that treats long-horizon multi-turn Text-to-SQL as an interactive MDP with execution feedback and memory-based coherence verification. It combines warm-start supervised fine-tuning on verified, long-horizon trajectories with end-to-end reinforcement learning guided by multi-level rewards that couple execution accuracy with memory-consistent reasoning. Across CoSQL and SParC, MTSQL-R1 achieves state-of-the-art performance with smaller backbones, driven by environment-driven verification and memory-guided refinement that address previous short-horizon limitations. The approach provides deeper insights into long-horizon capabilities and offers a practical path toward more reliable, executable, and coherent conversational SQL systems, with full code and logs slated for release after review.

Abstract

Multi-turn Text-to-SQL aims to translate a user's conversational utterances into executable SQL while preserving dialogue coherence and grounding to the target schema. However, most existing systems only regard this task as a simple text translation task and follow a short-horizon paradigm, generating a query per turn without execution, explicit verification, and refinement, which leads to non-executable or incoherent outputs. We present MTSQL-R1, an agentic training framework for long-horizon multi-turn Text-to-SQL. We cast the task as a Markov Decision Process (MDP) in which an agent interacts with (i) a database for execution feedback and (ii) a persistent dialogue memory for coherence verification, performing an iterative propose to execute -> verify -> refine cycle until all checks pass. Experiments on COSQL and SPARC demonstrate that MTSQL-R1 consistently outperforms strong baselines, highlighting the importance of environment-driven verification and memory-guided refinement for conversational semantic parsing. Full recipes (including code, trained models, logs, reasoning trajectories, etc.) will be released after the internal review to contribute to community research.

Figures (13)

• Figure 1: Comparison between existing methods and our MTSQL-R1 on the average of CoSQL and SParC benchmark. Our method outperforms both strong prompting-based and finetuned baselines, achieving superior performance across various model sizes.
• Figure 2: Short- vs. long-horizon modeling in multi-turn Text-to-SQL. Short-horizon models directly translate text to SQL (causing a large portion of execution error), while our long-horizon MTSQL-R1 interact with the database and the maintained dialogue memory for executable and consistent queries.
• Figure 3: Overview of the MTSQL-R1 training pipeline. (1) Phase 1 (Self-Taught warm-start SFT): MTSQL-R1 leverages verified multi-turn trajectories to provide initial supervision for warm-start fine-tuning. (2) Aha-moment trajectory: an illustrative long-horizon Text-to-SQL example generated by the final RL-trained model, shown to clarify the trajectory format. (3) Phase 2 (End-to-End long-horizon RL): the policy LLM interacts with the database and memory over multiple turns and is optimized with multi-turn RL to strengthen long-horizon reasoning.
• Figure 4: Difficulty-wise results: execution accuracy (\ref{['fig:acc-diff']}) and token length/interactions (\ref{['fig:token-diff']}) on CoSQL.
• Figure 5: The evolution of different Long-Horizon Abilities and related Execution Match performance from base model to RL model for Qwen3-1.7B.