Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TENSURE: Fuzzing Sparse Tensor Compilers (Registered Report)

Kabilan Mahathevan, Yining Zhang, Muhammad Ali Gulzar, Kirshanthan Sundararajah

Abstract

Sparse Tensor Compilers (STCs) have emerged as critical infrastructure for optimizing high-dimensional data analytics and machine learning workloads. The STCs must synthesize complex, irregular control flow for various compressed storage formats directly from high-level declarative specifications, thereby making them highly susceptible to subtle correctness defects. Existing testing frameworks, which rely on mutating computation graphs restricted to a standard vocabulary of operators, fail to exercise the arbitrary loop synthesis capabilities of these compilers. Furthermore, generic grammar-based fuzzers struggle to generate valid inputs due to the strict rules governing how indices are reused across multiple tensors. In this paper, we present TENSURE, the first extensible black-box fuzzing framework specifically designed for the testing of STCs. TENSURE leverages Einstein Summation (Einsum) notation as a general input abstraction, enabling the generation of complex, unconventional tensor contractions that expose corner cases in the code-generation phases of STCs. We propose a novel constraint-based generation algorithm that guarantees 100% semantic validity of synthesized kernels, significantly outperforming the ~3.3% validity rate of baseline grammar fuzzers. To enable metamorphic testing without a trusted reference, we introduce a set of semantic-preserving mutation operators that exploit algebraic commutativity and heterogeneity in storage formats. Our evaluation on two state-of-the-art systems, TACO and Finch, reveals widespread fragility, particularly in TACO, where TENSURE exposed crashes or silent miscompilations in a majority of generated test cases. These findings underscore the critical need for specialized testing tools in the sparse compilation ecosystem.

TENSURE: Fuzzing Sparse Tensor Compilers (Registered Report)

Abstract

Sparse Tensor Compilers (STCs) have emerged as critical infrastructure for optimizing high-dimensional data analytics and machine learning workloads. The STCs must synthesize complex, irregular control flow for various compressed storage formats directly from high-level declarative specifications, thereby making them highly susceptible to subtle correctness defects. Existing testing frameworks, which rely on mutating computation graphs restricted to a standard vocabulary of operators, fail to exercise the arbitrary loop synthesis capabilities of these compilers. Furthermore, generic grammar-based fuzzers struggle to generate valid inputs due to the strict rules governing how indices are reused across multiple tensors. In this paper, we present TENSURE, the first extensible black-box fuzzing framework specifically designed for the testing of STCs. TENSURE leverages Einstein Summation (Einsum) notation as a general input abstraction, enabling the generation of complex, unconventional tensor contractions that expose corner cases in the code-generation phases of STCs. We propose a novel constraint-based generation algorithm that guarantees 100% semantic validity of synthesized kernels, significantly outperforming the ~3.3% validity rate of baseline grammar fuzzers. To enable metamorphic testing without a trusted reference, we introduce a set of semantic-preserving mutation operators that exploit algebraic commutativity and heterogeneity in storage formats. Our evaluation on two state-of-the-art systems, TACO and Finch, reveals widespread fragility, particularly in TACO, where TENSURE exposed crashes or silent miscompilations in a majority of generated test cases. These findings underscore the critical need for specialized testing tools in the sparse compilation ecosystem.
Paper Structure (19 sections, 1 equation, 4 figures, 1 table, 3 algorithms)

This paper contains 19 sections, 1 equation, 4 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Tensors and Einsum
  4. Tensor Compilers and Loop Lowering
  5. Compressed Data Structure
  6. Sparse Tensor Compilers
  7. Automated Testing Techniques
  8. Motivation
  9. Design & Implementation
  10. Random Kernel Generation
  11. Mutation Operators
  12. Language-Agnostic Architecture
  13. Program Execution & Bug Detection
  14. Evaluation
  15. TACO Evaluation Results
  16. ...and 4 more sections

Figures (4)

  • Figure 2: Correct TACO-Generated Program for the Dense Case. Implements the kernel $A(j) = B(i, j) \cdot C(i)$ where $A$ is a dense vector, $B$ is a CSR matrix and $C$ is a sparse vector.
  • Figure 3: The main fuzzing loop. The system generates a random tensor program and a few mutated variants. Both versions are compiled and executed, and the outputs are compared to identify mismatches indicating compiler bugs.
  • Figure 4: Runtime Comparison: TACO vs Finch. The time axis uses a log scale to capture the disparity in compilation times.
  • Figure : TACO Generated Program (Buggy)