Routing End User Queries to Enterprise Databases

TL;DR

This work tackles routing natural language queries to multiple enterprise databases with heterogeneous schemas. It introduces two realistic benchmarks, Spider-Route and Bird-Route, created by merging train and test DBs to form a uniform repository and evaluating both in-domain and cross-domain settings. The authors propose a training-free, modular reasoning-based reranking strategy that combines embedding-based candidate retrieval with a four-step, LLM-assisted process to assess schema-based phrase mappings, coverage, and connectivity, and to compute a total score (coverage × connectivity) plus a semantic tie-breaker. Empirical results show state-of-the-art performance over strong baselines on both benchmarks, with robust behavior across domain similarities and notable gains from incorporating domain knowledge in Bird-Route. The work demonstrates that decomposing query routing into interpretable sub-tasks with explicit structural checks improves accuracy, scalability, and reliability in enterprise DB routing scenarios.

Abstract

We address the task of routing natural language queries in multi-database enterprise environments. We construct realistic benchmarks by extending existing NL-to-SQL datasets. Our study shows that routing becomes increasingly challenging with larger, domain-overlapping DB repositories and ambiguous queries, motivating the need for more structured and robust reasoning-based solutions. By explicitly modelling schema coverage, structural connectivity, and fine-grained semantic alignment, the proposed modular, reasoning-driven reranking strategy consistently outperforms embedding-only and direct LLM-prompting baselines across all the metrics.

Figures (2)

• Figure 1: Limitations of existing benchmarks and approaches addressed by our benchmark and approach
• Figure 2: Our Approach: Modular Reasoning Re-Ranking