Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mitigating Artifacts in Pre-quantization Based Scientific Data Compressors with Quantization-aware Interpolation

Pu Jiao, Sheng Di, Jiannan Tian, Mingze Xia, Xuan Wu, Yang Zhang, Xin Liang, Franck Cappello

TL;DR

Experiments demonstrate that the artifact mitigation algorithm can effectively improve the quality of decompressed data produced by pre-quantization based compressors while maintaining their high compression throughput.

Abstract

Error-bounded lossy compression has been regarded as a promising way to address the ever-increasing amount of scientific data in today's high-performance computing systems. Pre-quantization, a critical technique to remove sequential dependency and enable high parallelism, is widely used to design and develop high-throughput error-controlled data compressors. Despite the extremely high throughput of pre-quantization based compressors, they generally suffer from low data quality with medium or large user-specified error bounds. In this paper, we investigate the artifacts generated by pre-quantization based compressors and propose a novel algorithm to mitigate them. Our contributions are fourfold: (1) We carefully characterize the artifacts in pre-quantization based compressors to understand the correlation between the quantization index and compression error; (2) We propose a novel quantization-aware interpolation algorithm to improve the decompressed data; (3) We parallelize our algorithm in both shared-memory and distributed-memory environments to obtain high performance; (4) We evaluate our algorithm and validate it with two leading pre-quantization based compressors using five real-world datasets. Experiments demonstrate that our artifact mitigation algorithm can effectively improve the quality of decompressed data produced by pre-quantization based compressors while maintaining their high compression throughput.

Mitigating Artifacts in Pre-quantization Based Scientific Data Compressors with Quantization-aware Interpolation

TL;DR

Experiments demonstrate that the artifact mitigation algorithm can effectively improve the quality of decompressed data produced by pre-quantization based compressors while maintaining their high compression throughput.

Abstract

Error-bounded lossy compression has been regarded as a promising way to address the ever-increasing amount of scientific data in today's high-performance computing systems. Pre-quantization, a critical technique to remove sequential dependency and enable high parallelism, is widely used to design and develop high-throughput error-controlled data compressors. Despite the extremely high throughput of pre-quantization based compressors, they generally suffer from low data quality with medium or large user-specified error bounds. In this paper, we investigate the artifacts generated by pre-quantization based compressors and propose a novel algorithm to mitigate them. Our contributions are fourfold: (1) We carefully characterize the artifacts in pre-quantization based compressors to understand the correlation between the quantization index and compression error; (2) We propose a novel quantization-aware interpolation algorithm to improve the decompressed data; (3) We parallelize our algorithm in both shared-memory and distributed-memory environments to obtain high performance; (4) We evaluate our algorithm and validate it with two leading pre-quantization based compressors using five real-world datasets. Experiments demonstrate that our artifact mitigation algorithm can effectively improve the quality of decompressed data produced by pre-quantization based compressors while maintaining their high compression throughput.
Paper Structure (24 sections, 5 equations, 11 figures, 2 tables, 4 algorithms)

This paper contains 24 sections, 5 equations, 11 figures, 2 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Lossy Compression for Scientific Data
  4. Artifact Mitigation
  5. Background
  6. Pre-quantization in Scientific Data Compression
  7. Euclidean Distance Transform
  8. Overview
  9. Problem Formulation
  10. Design Overview
  11. Characterization of Pre-quantization Artifacts
  12. Quantization-aware Interpolation
  13. Parallelization
  14. Shared-memory Parallelization
  15. Distributed-memory parallelization
  16. ...and 9 more sections

Figures (11)

  • Figure 1: Overview of the proposed framework.
  • Figure 2: Visualization of errors in quantized data.
  • Figure 3: Workflow of the proposed quantization-aware interpolation and compensation using a 2D example. Step identifies quantization boundaries; step computes the first round EDT; step derives sign map and sign-flipping boundary; step computes the second round EDT; step performs interpolation and applies compensation.
  • Figure 4: Visualization of error slices for the three parallel methods. Original data, decompressed data and compensated decompressed data are shown on two sides respectively (visualization range is [-0.032, 0.032]).
  • Figure 5: Rate-distortion results with SSIM.
  • ...and 6 more figures