FCBench: Cross-Domain Benchmarking of Lossless Compression for Floating-Point Data

TL;DR

FCBench cross-links HPC and database communities by benchmarking 13 lossless floating-point compressors across 33 real-world datasets, using both CPU and GPU implementations and profiling with a roofline model. It combines compression performance with a simulated in-memory database workload to assess query overhead, yielding practical recommendations and a cross-domain map for compressor selection and development. The results reveal domain-dependent strengths: dictionary-based predictors excel on structured data, while delta/Lorenzo approaches fare better on HPC-like data; GPU implementations offer high throughput but contend with host-device transfer and branching costs. Overall, the work provides a rigorous framework and actionable guidance for deploying compression in integrated HPC-DB workflows, with implications for data layout, block sizing, and parallelization strategies.

Abstract

While both the database and high-performance computing (HPC) communities utilize lossless compression methods to minimize floating-point data size, a disconnect persists between them. Each community designs and assesses methods in a domain-specific manner, making it unclear if HPC compression techniques can benefit database applications or vice versa. With the HPC community increasingly leaning towards in-situ analysis and visualization, more floating-point data from scientific simulations are being stored in databases like Key-Value Stores and queried using in-memory retrieval paradigms. This trend underscores the urgent need for a collective study of these compression methods' strengths and limitations, not only based on their performance in compressing data from various domains but also on their runtime characteristics. Our study extensively evaluates the performance of eight CPU-based and five GPU-based compression methods developed by both communities, using 33 real-world datasets assembled in the Floating-point Compressor Benchmark (FCBench). Additionally, we utilize the roofline model to profile their runtime bottlenecks. Our goal is to offer insights into these compression methods that could assist researchers in selecting existing methods or developing new ones for integrated database and HPC applications.

Figures (11)

• Figure 1: Single-precision format of the IEEE 754 standard.
• Figure 2: The Lorenzo transform and hypercubes.
• Figure 3: Timeline of studied compression methods.
• Figure 4: Integrating HPC and database with HDF5 and Dataframes.
• Figure 5: BoxPlot of compression ratios.