DeepCQ: General-Purpose Deep-Surrogate Framework for Lossy Compression Quality Prediction

TL;DR

DeepCQ tackles the heavy cost of predicting compression quality for error-bounded lossy compressors in scientific datasets. It presents a general-purpose deep surrogate with a two-stage design: a shared Data Feature Extraction Network (DFE-NN) and a lightweight metric-prediction head consisting of an Error-bound Feature Extraction Network (EFE-NN) and a Prediction Network (Pred-NN), optionally augmented with a Mixture-of-Experts (MoE) to handle time-evolving data. This decouples feature extraction from prediction to reduce training overhead and enables modular inference across compressors and metrics such as PSNR, SSIM, and CR. Evaluations on four real-world datasets and five compressors show mean absolute percentage errors generally below $\le 0.10$, significantly outperforming prior approaches and enabling rapid compressor exploration. By enabling data-driven, quality-aware compression decisions, DeepCQ reduces I/O and computation overhead in exascale HPC workflows while preserving relevant scientific information.

Abstract

Error-bounded lossy compression techniques have become vital for scientific data management and analytics, given the ever-increasing volume of data generated by modern scientific simulations and instruments. Nevertheless, assessing data quality post-compression remains computationally expensive due to the intensive nature of metric calculations. In this work, we present a general-purpose deep-surrogate framework for lossy compression quality prediction (DeepCQ), with the following key contributions: 1) We develop a surrogate model for compression quality prediction that is generalizable to different error-bounded lossy compressors, quality metrics, and input datasets; 2) We adopt a novel two-stage design that decouples the computationally expensive feature-extraction stage from the light-weight metrics prediction, enabling efficient training and modular inference; 3) We optimize the model performance on time-evolving data using a mixture-of-experts design. Such a design enhances the robustness when predicting across simulation timesteps, especially when the training and test data exhibit significant variation. We validate the effectiveness of DeepCQ on four real-world scientific applications. Our results highlight the framework's exceptional predictive accuracy, with prediction errors generally under 10\% across most settings, significantly outperforming existing methods. Our framework empowers scientific users to make informed decisions about data compression based on their preferred data quality, thereby significantly reducing I/O and computational overhead in scientific data analysis.

Figures (7)

• Figure 1: The cumulative time to examine data quality across 10 error bounds. Dataset: Dark matter density; dimension:$[512 \times 512 \times 512]$.
• Figure 2: Similar pattern across four fields in the Hurricane application.
• Figure 3: Two-stage compression quality prediction framework. Highlighted parts are neural network components of our surrogate model.
• Figure 4: Compression quality of Baryon density across four timesteps.
• Figure 5: The architecture of Pred-NN. Highlighted components in b) represent the MoE design.