A Practical Theory of Generalization in Selectivity Learning

TL;DR

This paper addresses the theoretical underpinnings of query-driven selectivity learning in databases, aiming to reconcile practice (notably deep learning predictors) with theory that previously relied on probability measures and PAC guarantees. It introduces a generalization framework based on predictors induced by signed measures, proving in-distribution learnability with finite fat-dim and, under mild assumptions, out-of-distribution generalization bounds that scale with the square root of the in-distribution error. Building on this theory, the authors propose NeuroCDF, a CDF-modeling paradigm that is provably induced by signed measures, and SeConCDF, a training framework that enforces CDF self-consistency to improve OOD generalization while preserving in-distribution performance. Empirical results on single- and multi-table datasets show that SeConCDF substantially boosts OOD robustness and reduces query latency for NN-based models, without sacrificing in-distribution accuracy. The work suggests a viable path to combine theoretical guarantees with the practical strengths of neural models, and points to future directions such as extending beyond signed measures and exploring different loss functions like Qerror.

Abstract

Query-driven machine learning models have emerged as a promising estimation technique for query selectivities. Yet, surprisingly little is known about the efficacy of these techniques from a theoretical perspective, as there exist substantial gaps between practical solutions and state-of-the-art (SOTA) theory based on the Probably Approximately Correct (PAC) learning framework. In this paper, we aim to bridge the gaps between theory and practice. First, we demonstrate that selectivity predictors induced by signed measures are learnable, which relaxes the reliance on probability measures in SOTA theory. More importantly, beyond the PAC learning framework (which only allows us to characterize how the model behaves when both training and test workloads are drawn from the same distribution), we establish, under mild assumptions, that selectivity predictors from this class exhibit favorable out-of-distribution (OOD) generalization error bounds. These theoretical advances provide us with a better understanding of both the in-distribution and OOD generalization capabilities of query-driven selectivity learning, and facilitate the design of two general strategies to improve OOD generalization for existing query-driven selectivity models. We empirically verify that our techniques help query-driven selectivity models generalize significantly better to OOD queries both in terms of prediction accuracy and query latency performance, while maintaining their superior in-distribution generalization performance.

Figures (2)

• Figure 1: Left: three data points ($A,B,C$) and four ranges. Right Top: predictions from selectivity functions ($S_1 \sim S_4$) for the four ranges. Right Bottom: corresponding measures ($\mu_1 \sim \mu_4$) that induce $S_1 \sim S_4$, including their measure values on $A,B,C$ (technically, the three sets $\{A\}, \{B\}, \{C\}$).
• Figure 2: Left: relationship between a rectangle query and CDFs; Right: direct selectivity modeling v.s.$\textsc{NeuroCDF}$.

Theorems & Definitions (9)

• Example 3.1
• Lemma 3.1
• Theorem 4.1: In-Distribution Generalization Error Bound
• Example 4.1
• Theorem 4.2: OOD Generalization Error Bound
• Example 4.2: Center Move
• Example 4.3: Granularity Shift
• Proposition 4.1: OOD Generalization Error Bound with Data Shifts
• Theorem 5.1