GPU-Augmented OLAP Execution Engine: GPU Offloading

TL;DR

The paper addresses CPU bottlenecks in OLAP execution despite strong I/O improvements and proposes a GPU-augmented hybrid engine that offloads only high-impact primitives. It introduces a Risky Gate policy that gates GPU use based on $N$, transfer bytes $B$, candidate-set complexity $K$ and $M$, and estimated CPU versus GPU costs, using Key-Only transfers with late materialization to minimize data movement and improve tail latency. The approach is validated with PostgreSQL microbenchmarks and a GPU proxy, deriving a break-even model $T_{cpu}(N)=a\cdot N\log_2 N+b$ and $T_{tx}(N)=c\cdot N+d$ and solving $T_{cpu}(N^{*})=T_{tx}(N^{*})+T_{gpu}(N^{*})$, achieving alignment within 2.7% and up to a 12.9x end-to-end speedup. The work demonstrates that selective GPU offloading, guided by a tunable risk-aware policy and data-reduction techniques, yields robust performance gains in modern OLAP workloads with reduced tail latency and controlled data movement.

Abstract

Modern OLAP systems have mitigated I/O bottlenecks via storage-compute separation and columnar layouts, but CPU costs in the execution layer (especially Top-K selection and join probe) are emerging as new bottlenecks at scale. This paper proposes a hybrid architecture that augments existing vectorized execution by selectively offloading only high-impact primitives to the GPU. To reduce data movement, we use key-only transfer (keys and pointers) with late materialization. We further introduce a Risky Gate (risk-aware gating) that triggers offloading only in gain/risk intervals based on input size, transfer, kernel and post-processing costs, and candidate-set complexity (K, M). Using PostgreSQL microbenchmarks and GPU proxy measurements, we observe improved tail latency (P95/P99) under gated offloading compared to always-on GPU offloading. This work extends the risk-aware gating principle used for optimizer-stage GPU-assisted measurement (arXiv:2512.19750) to execution-layer OLAP primitives.

Figures (7)

• Figure 1: Risky Gate: Eliminating Small-N Overhead. The Risky Gate strategy avoids the latency penalty incurred by GPU offloading for small datasets.
• Figure 2: Gate Sensitivity: Offload Rate by Margin. The switching point shifts based on the defined margin, allowing for policy tuning.
• Figure 3: Engine Cost Scaling (Full Sort vs Top-K). Full Sort costs scale drastically ($N \log N$), whereas Top-K remains relatively low.
• Figure 4: Payload Size (Full-row vs Key-only). Key-only transfer significantly reduces the data payload.
• Figure 5: Measurement-driven break-even fitting and validation. The predicted break-even point closely matches the measured values.