DIVER: A Robust Text-to-SQL System with Dynamic Interactive Value Linking and Evidence Reasoning

TL;DR

DIVER introduces a training-free, multi-agent system for robust Text-to-SQL by automating evidence generation through Dynamic Interactive Value Linking and Evidence Reasoning. It uses a Break up Assistant to decompose NLQs, a Look up Assistant with a CoTF workspace and a toolbox to iteratively probe the database, and an Evidence Assistant to synthesize style-tailored evidence for downstream models. Extensive experiments on four benchmarks show up to 10.82% EX and 16.09% VES gains, with strong robustness improvements under dynamic database perturbations, outperforming expert-evidence baselines. The approach demonstrates that automated, adaptive value linking grounded in verified database facts substantially enhances real-world Text-to-SQL robustness and generalizability.

Abstract

In the era of large language models, Text-to-SQL, as a natural language interface for databases, is playing an increasingly important role. The sota Text-to-SQL models have achieved impressive accuracy, but their performance critically relies on expert-written evidence, which typically clarifies schema and value linking that existing models struggle to identify. Such limitations stem from the ambiguity of user queries and, more importantly, the complexity of comprehending large-scale and dynamic database values. Consequently, in real-world scenarios where expert assistance is unavailable, existing methods suffer a severe performance collapse, with execution accuracy dropping by over 10%. This underscores their lack of robustness. To address this, we propose DIVER, a robust system that automates evidence reasoning with dynamic interactive value linking. It leverages a compatible toolbox containing diverse tools to probe the database. Then, restricted by a structured workspace (CoTF, Chain of Thoughts and Facts), it reflects based on probe results and selects a new tool for next round of probing. Through this automatically iterative process, DIVER identifies schema and value linking missed by existing methods. Based on these accurate linkings, DIVER is able to infer correct usage of SQL functions and formulas and generate high-quality evidence, achieving robust Text-to-SQL without expert assistance. Extensive experiments demonstrate that: 1) The DIVER system significantly enhances the robustness of various Text-to-SQL models, improving performance by up to 10.82% in Execution Accuracy (EX) and 16.09% in Valid Efficiency Score (VES). 2) Our dynamic interactive value linking significantly improves the robustness of existing systems and the accuracy of schema and value linking, especially when confronted with challenges posed by large-scale, dynamic database values.

Figures (10)

• Figure 1: Existing Text-to-SQL methods utilize an offline expert-written evidence to aid the model, achieving high but unrealistic performance. They suffer a severe performance collapse in real-world scenario where expert is unavailable. DIVER system provides a training-free, online process that acts as an automated expert, generating evidence to achieve robust performance.
• Figure 2: Fine-grained analysis of expert-written evidence reveals that effective value linking is a critical prerequisite for robust Text-to-SQL performance. "VR" indicates the Value Retrieval module of methods. "VLE" represents expert-written value linking and schema linking evidence, and "SQLE" represents expert-written SQL-related Evidence such as function usage.
• Figure 3: An Example of Evidence and Golden SQL Overlap.
• Figure 4: Distribution of token overlap between expert-written evidence and the golden SQL query on the BIRD-dev dataset. The green plot shows significant answer leakage within the expert-written evidence, while the blue plot shows the degree to which this leakage directly contributes to the final SQL query.
• Figure 5: An Overview of DIVER Workflow.