Boundary-Aware NL2SQL: Integrating Reliability through Hybrid Reward and Data Synthesis

TL;DR

BAR-SQL tackles the reliability gap in NL2SQL for enterprise BI by integrating boundary awareness into the training loop. It combines Seed-Mutation Synthesis, Knowledge-Grounded Reasoning Synthesis (KGRS), and a Group Relative Policy Optimization (GRPO) framework guided by a Task-Condition-Aware Hybrid Reward (TCHR). A novel Ent-SQL-Bench benchmark is introduced to jointly assess SQL accuracy and abstention on ambiguous or unanswerable queries, where BAR-SQL achieves 91.48% average accuracy and outperforms proprietary baselines. The approach demonstrates a unified path to high-fidelity, interpretable SQL generation while maintaining responsible refusal behavior, with potential for deployment in data-driven enterprises and future enhancements via reasoning supervision and larger models.

Abstract

In this paper, we present BAR-SQL (Boundary-Aware Reliable NL2SQL), a unified training framework that embeds reliability and boundary awareness directly into the generation process. We introduce a Seed Mutation data synthesis paradigm that constructs a representative enterprise corpus, explicitly encompassing multi-step analytical queries alongside boundary cases including ambiguity and schema limitations. To ensure interpretability, we employ Knowledge-Grounded Reasoning Synthesis, which produces Chain-of-Thought traces explicitly anchored in schema metadata and business rules. The model is trained through a two-stage process: Supervised Fine-Tuning (SFT) followed by Reinforcement Learning via Group Relative Policy Optimization. We design a Task-Conditioned Hybrid Reward mechanism that simultaneously optimizes SQL execution accuracy-leveraging Abstract Syntax Tree analysis and dense result matching-and semantic precision in abstention responses. To evaluate reliability alongside generation accuracy, we construct and release Ent-SQL-Bench, which jointly assesse SQL precision and boundary-aware abstention across ambiguous and unanswerable queries. Experimental results on this benchmark demonstrate that BAR-SQL achieves 91.48% average accuracy, outperforming leading proprietary models, including Claude 4.5 Sonnet and GPT-5, in both SQL generation quality and boundary-aware abstention capability. The source code and benchmark are available anonymously at: https://github.com/TianSongS/BAR-SQL.

Figures (14)

• Figure 1: Seed data synthesis pipeline. The workflow begins with a dimension inventory containing dimensions and entities (TIME, METRICS, PROVINCE, CITIES, BRANDS, SERIES), which are used to construct 1,575 combinatorial query templates. Templates are instantiated via random sampling, then transformed into natural-language questions through LLM generation with Domain-KG-constrained analytical augmentation (e.g., Top-K ranking, extremum operations), ensuring all augmentations reference only schema-defined entities. The system performs consistency-verified LLM-augmented SQL synthesis by fusing background knowledge context (Schema KG, Metric KG, Domain KG, and Data KG). Generated SQLs undergo execution validation and human verification to produce human-verified triplets (Q, SQL, Result), eliminating empty results and erroneous queries. Finally, knowledge-grounded Chain-of-Thought (CoT) reasoning traces are synthesized through a 4-step process (Extract Entities, Logic Augment, Knowledge Retrieval, SQL Synthesis) to produce validated seed data triplets (Q, SQL, CoT).
• Figure 2: Overview of the BAR-SQL training pipeline. The framework consists of three stages: cold-start SFT with unified CoT format, TCHR reward function design (integrating Format, Grammar, Accuracy, and Length components), and GRPO-based policy optimization. The TCHR serves as the core feedback mechanism guiding the model to handle heterogeneous NL2SQL behaviors.
• Figure 3: Composition of the Task-Condition-Aware Hybrid Reward (TCHR) function. The reward aggregates four components—Format, Grammar, Accuracy, and Length—with task-adaptive weights. The Accuracy component bifurcates into Natural Language verification (using embedding-based similarity) and SQL verification (using execution matching with structural reward fallback). The structural reward combines AST-based similarity and dense execution-result matching to provide granular feedback when binary execution fails.
• Figure 4: Application pipeline of our NL2SQL system.
• Figure 5: Prompt for Natural-Language Question Generation with Emergent Augmentation