Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Implementing Semantic Join Operators Efficiently

Immanuel Trummer

TL;DR

This work addresses efficient semantic theta-joins in which the join predicate is expressed in natural language and evaluated by large language models. It introduces batch-based join operators inspired by block nested loops, accompanied by a formal token-budget cost model and adaptive selectivity mechanisms. The core contributions are a tuple-based baseline, a batch-oriented block join with explicit overflow handling, an analytic optimization of batch sizes under known selectivity, and an adaptive join that converges toward near-optimal costs without prior selectivity knowledge. Experimental results with GPT-4 show dramatic cost reductions and competitive accuracy compared with baselines like LOTUS across diverse data scenarios, enabling practical semantic joins at scale.

Abstract

Semantic query processing engines often support semantic joins, enabling users to match rows that satisfy conditions specified in natural language. Such join conditions can be evaluated using large language models (LLMs) that solve novel tasks without task-specific training. Currently, many semantic query processing engines implement semantic joins via nested loops, invoking the LLM to evaluate the join condition on row pairs. Instead, this paper proposes a novel algorithm, inspired by the block nested loops join operator implementation in traditional database systems. The proposed algorithm integrates batches of rows from both input tables into a single prompt. The goal of the LLM invocation is to identify all matching row pairs in the current input. The paper introduces formulas that can be used to optimize the size of the row batches, taking into account constraints on the size of the LLM context window (limiting both input and output size). An adaptive variant of the proposed algorithm refers to cases in which the size of the output is difficult to estimate. A formal analysis of asymptotic processing costs, as well as empirical results, demonstrates that the proposed approach reduces costs significantly and performs well compared to join implementations used by recent semantic query processing engines.

Implementing Semantic Join Operators Efficiently

TL;DR

This work addresses efficient semantic theta-joins in which the join predicate is expressed in natural language and evaluated by large language models. It introduces batch-based join operators inspired by block nested loops, accompanied by a formal token-budget cost model and adaptive selectivity mechanisms. The core contributions are a tuple-based baseline, a batch-oriented block join with explicit overflow handling, an analytic optimization of batch sizes under known selectivity, and an adaptive join that converges toward near-optimal costs without prior selectivity knowledge. Experimental results with GPT-4 show dramatic cost reductions and competitive accuracy compared with baselines like LOTUS across diverse data scenarios, enabling practical semantic joins at scale.

Abstract

Semantic query processing engines often support semantic joins, enabling users to match rows that satisfy conditions specified in natural language. Such join conditions can be evaluated using large language models (LLMs) that solve novel tasks without task-specific training. Currently, many semantic query processing engines implement semantic joins via nested loops, invoking the LLM to evaluate the join condition on row pairs. Instead, this paper proposes a novel algorithm, inspired by the block nested loops join operator implementation in traditional database systems. The proposed algorithm integrates batches of rows from both input tables into a single prompt. The goal of the LLM invocation is to identify all matching row pairs in the current input. The paper introduces formulas that can be used to optimize the size of the row batches, taking into account constraints on the size of the LLM context window (limiting both input and output size). An adaptive variant of the proposed algorithm refers to cases in which the size of the output is difficult to estimate. A formal analysis of asymptotic processing costs, as well as empirical results, demonstrates that the proposed approach reduces costs significantly and performs well compared to join implementations used by recent semantic query processing engines.

Paper Structure

This paper contains 20 sections, 15 theorems, 5 equations, 7 figures, 2 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Problem Model
  3. Tuple Nested Loops Join
  4. Algorithm
  5. Cost Model
  6. Block Nested Loops Join
  7. Algorithm
  8. Cost Model
  9. Optimizing for Known Selectivity
  10. Analyzing Costs
  11. Optimizing Costs
  12. Adaptive Join Algorithm
  13. Algorithm
  14. Analysis
  15. Experimental Results
  16. ...and 5 more sections

Key Result

lemma 1

Comparing two input tuples incurs cost $p+s_1+s_2+g$.

Figures (7)

  • Figure 1: Prompt template used for tuple nested loops join (instantiated by Function TuplePrompt in pseudo-code).
  • Figure 2: Prompt template used for block nested loops join (instantiated by Function BlockPrompt in pseudo-code).
  • Figure 3: Illustrating join processing costs as a function of the two input batch sizes ($b_1$ and $b_2$), using $r_1=50$, $r_2=10$, $s_1=10$, $s_2=2$, $s_3=1$, $\sigma=1$, $g=1$, $p=1$. All solutions under the white curve use prompts with a size at or below 100 tokens. The white X marks the solution with minimal cost among all solutions with a prompt size of up to 100 tokens.
  • Figure 4: Impact of selectivity $\sigma$ on optimal batch sizes and token allocations for $r_1=50$, $r_2=10$, $s_1=10$, $s_2=2$, $s_3=1$, $g=1$, $p=1$, and $t=100$.
  • Figure 5: Cost of simulated joins with GPT-4.
  • ...and 2 more figures

Theorems & Definitions (17)

  • definition 1: Semantic Join with Natural Language Predicates
  • definition 2: Large Language Model
  • lemma 1
  • corollary 1
  • lemma 2
  • lemma 3
  • lemma 4
  • corollary 2
  • theorem 1
  • lemma 5
  • ...and 7 more