AutoLink: Autonomous Schema Exploration and Expansion for Scalable Schema Linking in Text-to-SQL at Scale

TL;DR

AutoLink tackles the impracticality of provisioning large industrial schemas to LLMs for Text-to-SQL by treating schema linking as an autonomous, iterative agent problem guided by an LLM. It couples a dual-environment setup with semantic retrieval and lightweight SQL probing to grow a targeted linked schema $S_{linked}$ without exposing $S_{full}$, achieving high strict recall and competitive execution accuracy while dramatically reducing token costs. Through an action-driven loop with exploration, retrieval, verification, and expansion, plus a self-consistent SQL generation pipeline and majority voting, AutoLink demonstrates state-of-the-art SRR on Bird-Dev and Spider 2.0-Lite and strong scalability to databases with thousands of columns. The approach offers practical benefits for industrial Text-to-SQL systems by balancing recall, noise, and efficiency, enabling reliable SQL generation under tight context windows.

Abstract

For industrial-scale text-to-SQL, supplying the entire database schema to Large Language Models (LLMs) is impractical due to context window limits and irrelevant noise. Schema linking, which filters the schema to a relevant subset, is therefore critical. However, existing methods incur prohibitive costs, struggle to trade off recall and noise, and scale poorly to large databases. We present \textbf{AutoLink}, an autonomous agent framework that reformulates schema linking as an iterative, agent-driven process. Guided by an LLM, AutoLink dynamically explores and expands the linked schema subset, progressively identifying necessary schema components without inputting the full database schema. Our experiments demonstrate AutoLink's superior performance, achieving state-of-the-art strict schema linking recall of \textbf{97.4\%} on Bird-Dev and \textbf{91.2\%} on Spider-2.0-Lite, with competitive execution accuracy, i.e., \textbf{68.7\%} EX on Bird-Dev (better than CHESS) and \textbf{34.9\%} EX on Spider-2.0-Lite (ranking 2nd on the official leaderboard). Crucially, AutoLink exhibits \textbf{exceptional scalability}, \textbf{maintaining high recall}, \textbf{efficient token consumption}, and \textbf{robust execution accuracy} on large schemas (e.g., over 3,000 columns) where existing methods severely degrade-making it a highly scalable, high-recall schema-linking solution for industrial text-to-SQL systems.

Figures (9)

• Figure 1: Overview of AutoLink: iterative interactive agentic schema linking framework.
• Figure 2: Scalability comparison across databases on Spider 2.0-Lite of varying sizes in terms of (a) Strict Recall Rate (SRR$\uparrow$), (b) Average Tokens Consumption (Avg. Tokens$\downarrow$), and (c) Execution Accuracy (EX$\uparrow$). Percentages below each bin indicate the proportion of databases within each size range.
• Figure 3: SRR and EX comparison of schema linking at different initial top-$n$ on Spider 2.0-Lite and Bird dev set.
• Figure 4: Trade-off comparison between recall columns per instance and strict recall rate across different methods.
• Figure 5: The results on leaderboard of Spider 2.0-Lite. Among all the submitted methods, AutoLink ranked second.