Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Error Checking for Sparse Systolic Tensor Arrays

Christodoulos Peltekis, Dionysios Filippas, Giorgos Dimitrakopoulos

TL;DR

The paper tackles online error detection for structured-sparse tensor arrays used in ML accelerators by extending Algorithm-Based Fault Tolerance (ABFT) to sparse tensor architectures. It places peripheral checksum logic and reusable tensor PEs to compute actual and predicted checksums for the matrix product $C = A W$, employing a digit-serial arithmetic scheme to manage wider accumulators. Key contributions include a scalable ABFTChecksum architecture for sparse tensors, a digit-serial checksum approach, and hardware evaluations showing modest area overhead (<5%) and power overhead (~7–9%), along with strong fault-detection performance on ResNet50 CNN workloads. This work enables reliable sparse-structure ML inference in safety-critical settings with minimal impact on performance and energy efficiency.

Abstract

Structured sparsity is an efficient way to prune the complexity of modern Machine Learning (ML) applications and to simplify the handling of sparse data in hardware. In such cases, the acceleration of structured-sparse ML models is handled by sparse systolic tensor arrays. The increasing prevalence of ML in safety-critical systems requires enhancing the sparse tensor arrays with online error detection for managing random hardware failures. Algorithm-based fault tolerance has been proposed as a low-cost mechanism to check online the result of computations against random hardware failures. In this work, we address a key architectural challenge with structured-sparse tensor arrays: how to provide online error checking for a range of structured sparsity levels while maintaining high utilization of the hardware. Experimental results highlight the minimum hardware overhead incurred by the proposed checking logic and its error detection properties after injecting random hardware faults on sparse tensor arrays that execute layers of ResNet50 CNN.

Error Checking for Sparse Systolic Tensor Arrays

TL;DR

The paper tackles online error detection for structured-sparse tensor arrays used in ML accelerators by extending Algorithm-Based Fault Tolerance (ABFT) to sparse tensor architectures. It places peripheral checksum logic and reusable tensor PEs to compute actual and predicted checksums for the matrix product , employing a digit-serial arithmetic scheme to manage wider accumulators. Key contributions include a scalable ABFTChecksum architecture for sparse tensors, a digit-serial checksum approach, and hardware evaluations showing modest area overhead (<5%) and power overhead (~7–9%), along with strong fault-detection performance on ResNet50 CNN workloads. This work enables reliable sparse-structure ML inference in safety-critical settings with minimal impact on performance and energy efficiency.

Abstract

Structured sparsity is an efficient way to prune the complexity of modern Machine Learning (ML) applications and to simplify the handling of sparse data in hardware. In such cases, the acceleration of structured-sparse ML models is handled by sparse systolic tensor arrays. The increasing prevalence of ML in safety-critical systems requires enhancing the sparse tensor arrays with online error detection for managing random hardware failures. Algorithm-based fault tolerance has been proposed as a low-cost mechanism to check online the result of computations against random hardware failures. In this work, we address a key architectural challenge with structured-sparse tensor arrays: how to provide online error checking for a range of structured sparsity levels while maintaining high utilization of the hardware. Experimental results highlight the minimum hardware overhead incurred by the proposed checking logic and its error detection properties after injecting random hardware faults on sparse tensor arrays that execute layers of ResNet50 CNN.
Paper Structure (9 sections, 5 figures, 1 table)

This paper contains 9 sections, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background on Error checking on dense systolic arrays
  3. Online Error Checking for Sparse Systolic Tensor Arrays
  4. ABFT on Sparse Tensor Arrays
  5. Tackling the increased width of accumulators with digit-serial additions
  6. Experimental results
  7. Hardware complexity evaluation
  8. Evaluation of fault detection properties
  9. Conclusions

Figures (5)

  • Figure 1: Example of (a) unstructured sparsity; and (b) structured block sparsity of 2:4 (i.e., up to 2 non-zero elements in every 4 consecutive elements) and their respective packed storage with their associated bit masks.
  • Figure 2: A typical systolic array for a fully dense matrix multiplication following the weight-stationary (WS) dataflow.
  • Figure 3: A two-row sparse systolic tensor array that can be configured for $2$:$4$ and $1$:$4$ sparsity patterns. ABFT error checking that computes the actual and the predicted checksum is placed at the periphery of the array.
  • Figure 4: Checksum computation follows back-to-back normal computation. Checksums are computed in a digit-serial manner reusing the fixed structured of the sparse tensor array.
  • Figure 5: The total (a) area and (b) power of sparse tensor arrays for various sizes and for 2:4 and 1:4 sparsity patterns. Both diagrams highlight separately the contribution of the checker's logic.