SySTeC: A Symmetric Sparse Tensor Compiler

TL;DR

SySTeC tackles the challenging problem of simultaneously exploiting symmetry and sparsity in tensor kernels by introducing a two-phase compiler workflow that first symmetrizes computations to operate on the canonical triangle and then applies a suite of optimizations to eliminate redundancy. The approach is grounded in a formal taxonomy of symmetry (input/output, visible/invisible) and leverages Finch IR to realize symmetry-aware, sparse-tensor code generation. Empirical results across SSYMV, SYPRD, SSYRK, TTM, and MTTKRP demonstrate substantial speedups over non-symmetric baselines, with the largest gains occurring in high-dimensional MTTKRP (up to $29.8\times$). The work provides a practical pathway to high-performance symmetric-sparse tensor computation and offers reusable compiler techniques (e.g., diagonal splitting, simplicial lookup tables) that extend beyond the studied kernels.

Abstract

Symmetric and sparse tensors arise naturally in many domains including linear algebra, statistics, physics, chemistry, and graph theory. Symmetric tensors are equal to their transposes, so in the $n$-dimensional case we can save up to a factor of $n!$ by avoiding redundant operations. Sparse tensors, on the other hand, are mostly zero, and we can save asymptotically by processing only nonzeros. Unfortunately, specializing for both symmetry and sparsity at the same time is uniquely challenging. Optimizing for symmetry requires consideration of $n!$ transpositions of a triangular kernel, which can be complex and error prone. Considering multiple transposed iteration orders and triangular loop bounds also complicates iteration through intricate sparse tensor formats. Additionally, since each combination of symmetry and sparse tensor formats requires a specialized implementation, this leads to a combinatorial number of cases. A compiler is needed, but existing compilers cannot take advantage of both symmetry and sparsity within the same kernel. In this paper, we describe the first compiler which can automatically generate symmetry-aware code for sparse or structured tensor kernels. We introduce a taxonomy for symmetry in tensor kernels, and show how to target each kind of symmetry. Our implementation demonstrates significant speedups ranging from 1.36x for SSYMV to 30.4x for a 5-dimensional MTTKRP over the non-symmetric state of the art.

Figures (11)

• Figure 1: Finch Syntax ahrens_finch_2024
• Figure 2: On left, a naive SSYMV. On right, SSYMV that accesses only canonical triangle and reuses memory reads.
• Figure 3: Pseudocode for Symmetrized Kernel
• Figure 4: Symmetric Compiler Flow
• Figure 5: MTTKRP Symmetrization: We construct the unique symmetry groups given each equivalence group.

Theorems & Definitions (6)

• definition 1: Symmetry
• definition 2: Partial Symmetry
• definition 3: Canonical
• definition 4: Diagonal
• definition 5: Equivalence Group
• definition 6: Unique Symmetry Group