Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Birdie: Natural Language-Driven Table Discovery Using Differentiable Search Index

Yuxiang Guo, Zhonghao Hu, Yuren Mao, Baihua Zheng, Yunjun Gao, Mingwei Zhou

TL;DR

Birdie tackles NL-driven table discovery by introducing a differentiable search index that unifies indexing and search in a single encoder-decoder model. It learns prefix-aware tabids through a two-view clustering of tabular metadata and content, and uses an LLM-based generator with table-sampling to create high-quality synthetic queries for training. The framework supports continual indexing with parameter isolation to mitigate catastrophic forgetting, and experiments show significant accuracy gains over state-of-the-art dense methods plus strong continual-learning performance. The approach yields practical benefits for large, dynamic table repositories by delivering fast, end-to-end NL-to-table retrieval and robust updates, with potential for scalable deployment in data lakes and enterprise data ecosystems.

Abstract

Natural language (NL)-driven table discovery identifies relevant tables from large table repositories based on NL queries. While current deep-learning-based methods using the traditional dense vector search pipeline, i.e., representation-index-search, achieve remarkable accuracy, they face several limitations that impede further performance improvements: (i) the errors accumulated during the table representation and indexing phases affect the subsequent search accuracy; and (ii) insufficient query-table interaction hinders effective semantic alignment, impeding accuracy improvements. In this paper, we propose a novel framework Birdie, using a differentiable search index. It unifies the indexing and search into a single encoder-decoder language model, thus getting rid of error accumulations. Birdie first assigns each table a prefix-aware identifier and leverages a large language model-based query generator to create synthetic queries for each table. It then encodes the mapping between synthetic queries/tables and their corresponding table identifiers into the parameters of an encoder-decoder language model, enabling deep query-table interactions. During search, the trained model directly generates table identifiers for a given query. To accommodate the continual indexing of dynamic tables, we introduce an index update strategy via parameter isolation, which mitigates the issue of catastrophic forgetting. Extensive experiments demonstrate that Birdie outperforms state-of-the-art dense methods by 16.8% in accuracy, and reduces forgetting by over 90% compared to other continual learning approaches.

Birdie: Natural Language-Driven Table Discovery Using Differentiable Search Index

TL;DR

Birdie tackles NL-driven table discovery by introducing a differentiable search index that unifies indexing and search in a single encoder-decoder model. It learns prefix-aware tabids through a two-view clustering of tabular metadata and content, and uses an LLM-based generator with table-sampling to create high-quality synthetic queries for training. The framework supports continual indexing with parameter isolation to mitigate catastrophic forgetting, and experiments show significant accuracy gains over state-of-the-art dense methods plus strong continual-learning performance. The approach yields practical benefits for large, dynamic table repositories by delivering fast, end-to-end NL-to-table retrieval and robust updates, with potential for scalable deployment in data lakes and enterprise data ecosystems.

Abstract

Natural language (NL)-driven table discovery identifies relevant tables from large table repositories based on NL queries. While current deep-learning-based methods using the traditional dense vector search pipeline, i.e., representation-index-search, achieve remarkable accuracy, they face several limitations that impede further performance improvements: (i) the errors accumulated during the table representation and indexing phases affect the subsequent search accuracy; and (ii) insufficient query-table interaction hinders effective semantic alignment, impeding accuracy improvements. In this paper, we propose a novel framework Birdie, using a differentiable search index. It unifies the indexing and search into a single encoder-decoder language model, thus getting rid of error accumulations. Birdie first assigns each table a prefix-aware identifier and leverages a large language model-based query generator to create synthetic queries for each table. It then encodes the mapping between synthetic queries/tables and their corresponding table identifiers into the parameters of an encoder-decoder language model, enabling deep query-table interactions. During search, the trained model directly generates table identifiers for a given query. To accommodate the continual indexing of dynamic tables, we introduce an index update strategy via parameter isolation, which mitigates the issue of catastrophic forgetting. Extensive experiments demonstrate that Birdie outperforms state-of-the-art dense methods by 16.8% in accuracy, and reduces forgetting by over 90% compared to other continual learning approaches.
Paper Structure (26 sections, 2 theorems, 7 equations, 14 figures, 10 tables, 3 algorithms)

This paper contains 26 sections, 2 theorems, 7 equations, 14 figures, 10 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Overview of Birdie
  4. Indexing from Scratch
  5. Two-view-based Tabid Assignment
  6. LLM-based Query Generation
  7. Model Training and Inference
  8. Index Update
  9. Incremental Tabid Assignment
  10. Continual Indexing via Parameter Isolation
  11. Experiments
  12. Experimental Setup
  13. Evaluation of Indexing from Scratch
  14. Ablation Study
  15. Evaluation of Index Update
  16. ...and 11 more sections

Key Result

Lemma 1

The upper and lower bounds of the radius $r'$ are given by $r + \operatorname{dist}(\mathbf{c}, \mathbf{c}^\prime)$ and $\operatorname{dist}(\mathbf{h}_{new}, \mathbf{c}^\prime)$, respectively.

Figures (14)

  • Figure 1: An example of NL-driven table discovery.
  • Figure 2: Traditional dense pipeline vs. DSI pipeline.
  • Figure 3: The framework of Birdie.
  • Figure 4: An example of a two-view-based clustering tree.
  • Figure 5: Instruction data for query generator training.
  • ...and 9 more figures

Theorems & Definitions (5)

  • Definition 1
  • Definition 2
  • Example 1
  • Lemma 1
  • Lemma 1