Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fast Topology-Aware Lossy Data Compression with Full Preservation of Critical Points and Local Order

Alex Fallin, Nathaniel Gorski, Tripti Agarwal, Bei Wang, Ganesh Gopalakrishnan, Martin Burtscher

Abstract

Many scientific codes and instruments generate large amounts of floating-point data at high rates that must be compressed before they can be stored. Typically, only lossy compression algorithms deliver high-enough compression ratios. However, many of them provide only point-wise error bounds and do not preserve topological aspects of the data such as the relative magnitude of neighboring points. Even topology-preserving compressors tend to merely preserve some critical points and are generally slow. Our Local-Order-Preserving Compressor is the first to preserve the full local order (and thus all critical points), runs orders of magnitude faster than prior topology-preserving compressors, yields higher compression ratios than lossless compressors, and produces bit-for-bit the same output on CPUs and GPUs.

Fast Topology-Aware Lossy Data Compression with Full Preservation of Critical Points and Local Order

Abstract

Many scientific codes and instruments generate large amounts of floating-point data at high rates that must be compressed before they can be stored. Typically, only lossy compression algorithms deliver high-enough compression ratios. However, many of them provide only point-wise error bounds and do not preserve topological aspects of the data such as the relative magnitude of neighboring points. Even topology-preserving compressors tend to merely preserve some critical points and are generally slow. Our Local-Order-Preserving Compressor is the first to preserve the full local order (and thus all critical points), runs orders of magnitude faster than prior topology-preserving compressors, yields higher compression ratios than lossless compressors, and produces bit-for-bit the same output on CPUs and GPUs.

Paper Structure

This paper contains 20 sections, 4 figures, 9 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Related Work
  4. Lossless Compressors
  5. Non-topology Preserving Lossy Compressors
  6. Topology Preserving Lossy Compressors
  7. LOPC Algorithm and Implementation
  8. Quantization
  9. Preserving Critical Points and Local Order
  10. Compression
  11. Parallelization and Optimization
  12. Correctness and Termination
  13. Experimental Methodology
  14. Results
  15. Critical-Point and Local-Order Preservation
  16. ...and 5 more sections

Figures (4)

  • Figure 1: Example of bit-shuffling lossless stage; for larger inputs, the sequences of bits with the same color are longer
  • Figure 2: Example of zero-byte elimination lossless stage; further compression of the bitmap is not shown
  • Figure 3: Geometric-mean compression ratio and compression runtime of LOPC on 7 NOA error bounds.
  • Figure 4: Average portion of the compressed file that is bin data and subbin data for LOPC on 7 NOA error bounds.