Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reflective Reasoning for SQL Generation

Isabelle Mohr, Joao Gandarela, John Dujany, Andre Freitas

TL;DR

A controlled text-to-SQL framework built around reflective refinement, which combines interpreter-based checks with LLM-based semantic coverage verification as epistemic judges and demonstrates consistent gains over strong prompting baselines, robust convergence within a small refinement budget, and improved execution accuracy across both frontier and open-weight model families.

Abstract

Robust text-to-SQL over complex, real-world databases remains brittle even with modern LLMs: iterative refinement often introduces syntactic and semantic drift, corrections tend to be non-transferable across queries, and naive use of large context windows scales poorly. We propose a controlled text-to-SQL framework built around reflective refinement. Instead of repeatedly rewriting the current SQL instance, the system decomposes generation into typed stages and applies feedback as persistent updates to the stage-level generation mechanism. A Reflection-Refinement Loop localizes violations to the responsible stage maximize preservation of previously validated constraints and support monotonic improvement over a query set. The method operates without gold SQL by combining interpreter-based checks with LLM-based semantic coverage verification as epistemic judges. Experiments on Spider and BIRD demonstrate consistent gains over strong prompting baselines, robust convergence within a small refinement budget, and improved execution accuracy across both frontier and open-weight model families.

Reflective Reasoning for SQL Generation

TL;DR

A controlled text-to-SQL framework built around reflective refinement, which combines interpreter-based checks with LLM-based semantic coverage verification as epistemic judges and demonstrates consistent gains over strong prompting baselines, robust convergence within a small refinement budget, and improved execution accuracy across both frontier and open-weight model families.

Abstract

Robust text-to-SQL over complex, real-world databases remains brittle even with modern LLMs: iterative refinement often introduces syntactic and semantic drift, corrections tend to be non-transferable across queries, and naive use of large context windows scales poorly. We propose a controlled text-to-SQL framework built around reflective refinement. Instead of repeatedly rewriting the current SQL instance, the system decomposes generation into typed stages and applies feedback as persistent updates to the stage-level generation mechanism. A Reflection-Refinement Loop localizes violations to the responsible stage maximize preservation of previously validated constraints and support monotonic improvement over a query set. The method operates without gold SQL by combining interpreter-based checks with LLM-based semantic coverage verification as epistemic judges. Experiments on Spider and BIRD demonstrate consistent gains over strong prompting baselines, robust convergence within a small refinement budget, and improved execution accuracy across both frontier and open-weight model families.
Paper Structure (34 sections, 6 equations, 3 figures, 5 tables, 1 algorithm)

This paper contains 34 sections, 6 equations, 3 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. The Phenomena: Why is text-to-SQL an intrinsically a hard problem?
  3. (Challenge 1)
  4. (Challenge 2)
  5. (Challenge 3)
  6. The Mechanism: Reflective self-correcting reasoning over localized feedback.
  7. Preliminaries
  8. Reflective refinement
  9. Outline.
  10. Reflective refinement.
  11. DB Context Interpretation
  12. Experiments
  13. Setup
  14. Benchmarks.
  15. Baselines.
  16. ...and 19 more sections

Figures (3)

  • Figure 1: High-level overview of the reflective text-to-SQL approach, showing staged generation, critic feedback, semantic checking, and iterative refinement.
  • Figure 2: Early refinement yields the majority of gains across all models, while excessive iterations lead to diminishing returns or over-refinement, motivating selective and bounded refinement. EX is reported at each refinement step $t$ for a range of base models on Spider 1.0 (dev) and BIRD (dev).
  • Figure 3: Illustrative example of selective reflective refinement in the text-to-QL pipeline. The diagram shows an end-to-end inference trace for a query involving an out-of-range laboratory constraint and a superlative aggregation. An initial compositional pass produces an incorrect SQL realization. Epistemic judges and a critic localize the failure to the SQL realization component, triggering a targeted refinement of its generation parameters. The refined realization enforces correct predicate polarity and aggregation grouping, yielding a syntactically and semantically correct final query.