Fast Entropy Decoding for Sparse MVM on GPUs

TL;DR

dtANS, the new lossless compression method that improves the entropy coding technique of asymmetric numeral systems (ANS) specifically for fast parallel GPU decoding when used in tandem with SpMVM, is applied on the widely used CSR format.

Abstract

We present a novel, practical approach to speed up sparse matrix-vector multiplication (SpMVM) on GPUs. The novel key idea is to apply lossless entropy coding to further compress the sparse matrix when stored in one of the commonly supported formats. Our method is based on dtANS, our new lossless compression method that improves the entropy coding technique of asymmetric numeral systems (ANS) specifically for fast parallel GPU decoding when used in tandem with SpMVM. We apply dtANS on the widely used CSR format and present extensive benchmarks on the SuiteSparse collection of matrices against the state-of-the-art cuSPARSE library. On matrices with at least 2^(15) entries and at least 10 entries per row on average, our compression reduces the matrix size over the smallest cuSPARSE format (CSR, COO and SELL) in almost all cases and up to 11.77 times. Further, we achieve an SpMVM speedup for the majority of matrices with at least 2^(25) nonzero entries. The best speedup is 3.48x. We also show that we can improve over the AI-based multi-format AlphaSparse in an experiment that is limited due to its extreme computation overhead. We provide our code as an open source C++/CUDA header library, which includes both compression and multiplication kernels.

Figures (9)

• Figure 1: Encoding into CSR-dtANS (left) and SpMVM on CSR-dtANS (right)
• Figure 2: Example for the CSR format with 6 nonzeros
• Figure 3: Example of entropy encoding with tANS
• Figure 4: Entropy reduction via delta-encoding for three random graph models with increasing number of nodes. Model parameters are chosen to keep the average degree at 5, 10, and 20.
• Figure 5: Mapping of data to threads and dtANS representation