ATC: an Advanced Tucker Compression library for multidimensional data

TL;DR

ATC introduces a advanced Tucker-based lossy compressor for dense tensors, integrating ST-HOSVD rank truncation with TTHRESH-style bit-plane quantization to achieve high compression rates. By blending hybrid truncation, parallelized quantization, and improved error control via dequantization corrections, ATC delivers substantial speedups and memory savings while maintaining precise target-error fidelity. Across diverse datasets, ATC matches or exceeds state-of-the-art Tucker-based compressors, particularly in high-error regimes, and demonstrates competitive rate-distortion performance against non-Tucker methods. The work highlights practical benefits for large-scale multidimensional data workflows, including flexible interfaces, robust usability features, and strong potential for integration in scientific computing pipelines.

Abstract

We present ATC, a C++ library for advanced Tucker-based lossy compression of dense multidimensional numerical data in a shared-memory parallel setting, based on the sequentially truncated higher-order singular value decomposition (ST-HOSVD) and bit plane truncation. Several techniques are proposed to improve speed, memory usage, error control and compression rate. First, a hybrid truncation scheme is described which combines Tucker rank truncation and TTHRESH quantization [Ballester-Ripoll et al., IEEE Trans. Visual. Comput. Graph., 2020]. We derive a novel expression to approximate the error of truncated Tucker decompositions in the case of core and factor perturbations. Furthermore, we parallelize the quantization and encoding scheme and adjust this phase to improve error control. Moreover, implementation aspects are described, such as an ST-HOSVD procedure using only a single transposition. We also discuss several usability features of ATC, including the presence of multiple interfaces, extensive data type support and integrated downsampling of the decompressed data. Numerical results show that ATC maintains state-of-the-art Tucker compression rates, while providing average speed-up factors of 2.2-3.5 and halving memory usage. Furthermore, our compressor provides precise error control, only deviating 1.4% from the requested error on average. Finally, ATC often achieves higher compression than non-Tucker-based compressors in the high-error domain.

Figures (9)

• Figure 1: A sample of spectral slices from the Moffett-Field hyperspectral image (indices are 0-based). For visibility, a different scaling factor was used for each visualization, yet all slices clearly exhibit the same structure, showing great potential for compression.
• Figure 2: ATC compression examples using the Isotropic-PT dataset. Each visualization only shows the first time slice of the data tensor, while the statistics in the captions represent the full data.
• Figure 3: Overview of the ATC compression pipeline. The blue components (with section labels 4.1.1, 4.1.2, 4.1.3, 4.2 and 4.3) indicate the most important parts of the pipeline that were added or modified with respect to TTHRESH and are described in the corresponding sections. The green components (with section labels S.2, S.3, S.4, S.5 and S.6) only represent minor performance improvements and are therefore described in the supplementary material.
• Figure 4: Relative compression factors across different errors and datasets in terms of RTMSS. Each compression factor is normalized by dividing it by the maximum compression factor for the same error. The dashed line indicates the default RTMSS value, which balances suboptimal compression rates with increased compression and decompression speed.
• Figure 5: Rate-distortion curves for all compressors and datasets, both in terms of absolute compression factors and the compression factors relative to ATC.