Cost-based Selection of Provenance Sketches for Data Skipping
Ziyu Liu, Boris Glavic
TL;DR
The paper tackles the challenge of selecting effective provenance sketches to accelerate data-skipping in provenance-based data skipping (PBDS). It introduces a sample-based size estimation framework rooted in approximate query processing to predict the footprint of sketches built on candidate range-partition attributes, enabling principled, cost-based sketch selection. By combining safety checks, stratified sampling, bootstrap-based confidence intervals, and multiple candidate-attribute strategies, the approach consistently ranks and selects sketches that minimize data coverage while preserving query correctness. Implemented within a PBDS prototype on PostgreSQL, the method yields accurate size estimates and near-optimal sketch choices, achieving substantial runtime improvements (up to 60%) across diverse workloads and datasets. The results demonstrate the practicality of online sketch selection and provide a roadmap for scalable PBDS deployment in real-world DBMSs.
Abstract
Provenance sketches, light-weight indexes that record what data is needed (is relevant) for answering a query, can significantly improve performance of important classes of queries (e.g., HAVING and top-k queries). Given a horizontal partition of a table, a provenance sketch for a query Q records which fragments contain provenance. Once a provenance sketch has been captured for a query, it can be used to speed-up subsequent queries by skipping data that does not belong to a sketch. The size and, thus, also the effectiveness of a provenance sketch is often quite sensitive to the choice of attribute(s) we are partitioning on. In this work, we develop sample-based estimation techniques for the size of provenance sketches akin to a specialized form of approximate query processing. This technique enables the online selection of provenance sketches by estimating the size of sketches for a set of candidate attributes and then creating the sketch that is estimated to yield the largest benefit. We demonstrate experimentally that our estimation is accurate enough to select optimal or near optimal provenance sketches in most cases which in turn leads to a runtime improvement of up to %60 compared to other strategies for selecting provenance sketches.