Making Databases Faster with LLM Evolutionary Sampling

TL;DR

This work tackles OLAP query performance by using LLMs to refine physical query plans at test time. It introduces DBPlanBench, a harness that exposes DataFusion's physical plans in a compact, patch-editable form and uses evolutionary sampling to guide semantic, correctness-preserving plan edits. The approach leverages semantic cardinality estimation and join-optimization strategies to yield substantial speedups (up to 4.78x) and demonstrates that optimizations discovered on small scales transfer to larger databases. A rigorous evaluation protocol with sandboxed executions and deterministic plan transfer substantiates the practicality and robustness of LLM-driven plan refinement for real-world database workloads.

Abstract

Traditional query optimization relies on cost-based optimizers that estimate execution cost (e.g., runtime, memory, and I/O) using predefined heuristics and statistical models. Improving these heuristics requires substantial engineering effort, and even when implemented, these heuristics often cannot take into account semantic correlations in queries and schemas that could enable better physical plans. Using our DBPlanBench harness for the DataFusion engine, we expose the physical plan through a compact serialized representation and let the LLM propose localized edits that can be applied and executed. We then apply an evolutionary search over these edits to refine candidates across iterations. Our key insight is that LLMs can leverage semantic knowledge to identify and apply non-obvious optimizations, such as join orderings that minimize intermediate cardinalities. We obtain up to 4.78$\times$ speedups on some queries and we demonstrate a small-to-large workflow in which optimizations found on small databases transfer effectively to larger databases.

Figures (14)

• Figure 1: Semantic-aware Join Optimization. By identifying the high selectivity of the 'date' predicate, the plan can be reordered to prune data early, significantly reducing the cardinality of the subsequent JOIN. We explore this use case in Section \ref{['case-study']}.
• Figure 2: DBPlanBench, provides accessible APIs to extract a physical plan that an LLM can edit. This plan can be then executed in DataFusion as if it was a native DataFusion plan.
• Figure 3: Visualization of the evolutionary algorithms applied.
• Figure 4: CCDF Plots Representing the Change in the Distribution Throughout the Evolution.BoL-Evol results are plotted for TPC-H and PST-Evol results are plotted for the TPC-DS. The y-axis ($\text{fast}_x(t)$) is in the log-scale. We observe diminishing returns when the step is larger. Moreover, evolutionary algorithms help pushing the frontier in achieving higher speedups in contrast to base sampling.
• Figure 5: Plan-transfer generalization across scale factors. Scatter of speedup at large scale vs. small scale under the new method. The diagonal indicates parity ($y=x$).