Bridging Global Intent with Local Details: A Hierarchical Representation Approach for Semantic Validation in Text-to-SQL

TL;DR

This work tackles semantic validation in Text-to-SQL by bridging global intent and local structural details. It introduces HeroSQL, a hierarchical representation that combines a Logical Plan for global semantics with Abstract Syntax Trees for fine-grained syntactic structure, augmented by a Nested Message Passing Neural Network to propagate schema-guided information. To address data scarcity for fine-grained validation, it adds an adaptive sub-SQL augmentation strategy using AST perturbations and LLM-based generation, producing robust, semantically challenging negatives. Empirical results across multiple benchmarks show substantial improvements in AUPRC and AUROC, with strong generalization to unseen domains and tangible benefits for error correction and feedback to LLM-based generation. Overall, HeroSQL enhances reliability and interpretability of Text-to-SQL systems by providing fine-grained semantic validation and actionable feedback signals for query optimization.

Abstract

Text-to-SQL translates natural language questions into SQL statements grounded in a target database schema. Ensuring the reliability and executability of such systems requires validating generated SQL, but most existing approaches focus only on syntactic correctness, with few addressing semantic validation (detecting misalignments between questions and SQL). As a consequence, effective semantic validation still faces two key challenges: capturing both global user intent and SQL structural details, and constructing high-quality fine-grained sub-SQL annotations. To tackle these, we introduce HEROSQL, a hierarchical SQL representation approach that integrates global intent (via Logical Plans, LPs) and local details (via Abstract Syntax Trees, ASTs). To enable better information propagation, we employ a Nested Message Passing Neural Network (NMPNN) to capture inherent relational information in SQL and aggregate schema-guided semantics across LPs and ASTs. Additionally, to generate high-quality negative samples, we propose an AST-driven sub-SQL augmentation strategy, supporting robust optimization of fine-grained semantic inconsistencies. Extensive experiments conducted on Text-to-SQL validation benchmarks (both in-domain and out-of-domain settings) demonstrate that our approach outperforms existing state-of-the-art methods, achieving an average 9.40% improvement of AUPRC and 12.35% of AUROC in identifying semantic inconsistencies. It excels at detecting fine-grained semantic errors, provides large language models with more granular feedback, and ultimately enhances the reliability and interpretability of data querying platforms.

Figures (5)

• Figure 1: Syntactic errors can be easily detected and fixed, while executable SQL with semantic errors is hard to discover.
• Figure 2: The overall framework of HeroSQL. Given a natural language question, database schema information, and a syntactically correct SQL query generated by an LLM, HeroSQL first applies a query optimizer to convert the query into a logical plan (LP) and parses each LP node into an abstract syntax tree (AST). The AST nodes and contextual information are then encoded into property embeddings using a pretrained LLM-based model. Finally, these embeddings are aggregated by NMPNN to form a SQL embedding, which, together with the question embedding, is fed into an MLP for semantic validation and correctness prediction.
• Figure 3: Visualization of class distribution of test sets across datasets.
• Figure 4: Sensitivity analysis of layer number $l$ in nested message passing neural network
• Figure 9: Class distribution of test sets across datasets